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CHAIN POOLING TO MINIMIZE PREDICTION ERRORS IN SUBSET REGRESSION 


by Arthur G. Holms 
Lewis Research Center 


SUMMARY 

A technique called chain pooling had been developed for the analysis 
of results of two-level fixed effects full or fractional factorial exper- 
iments not having replication. The basic strategy includes the use of one 
nominal level of significance for some preliminary tests and a second nom- 
inal level of significance for the final tests. Strategies were identi- 
fied having approximate optimality with respect to the probabilities of 
Type 1 or Type 2 errors. 

The subject has been reexamined from the point of view of minimizing 
prediction errors in the resulting subset regression equations. The in- 
vestigation consisted of Monte Carlo studies using population models hav- 
ing geometries chosen to represent response surface applications, as moti- 
vated by some research and development in structural materials optimiza- 
tion. Parameter values were chosen to be unfavorable to the decision 
procedures. Simulated experiments were generated by adding pseudo norm- 
ally distributed random errors to population values to generate "observa- 
tions. 11 Model equations were fitted to the "observations" and the deci- 
sion procedures were used to delete terms. Comparison of values pre- 
dicted by the reduced models with population values enabled the identi- 
fication of a deletion strategy that is approximately optimal for minimiz- 
ing prediction errors. 

The results are proposed for any situation giving orthogonal esti- 
mators of the model coefficients. They include the cases of orthogonal 
designs of experiments and or thogonalizing transformations of terms of 
the initial regression model. 


INTRODUCTION 

In the development of materials for aerospace applications, the ex- 
perimentation is often expensive and time consuming. In seeking optimum 
processing conditions or optimum compositions, the experimentation is 
usually done with costly raw materials and elaborate processes. The tests 
involved can include creep, stress rupture, fatigue, or other environ- 
mental simulations and can be time consuming. Furthermore, the complexity 
of the total fabrication and testing process permits the random cumulative 
effect of important levels of experimental error. The efficient procedure 
for mitigating the effects of such error, while minimizing the costs of 
experimentation, has been to practice mathematical modeling of the data. 
Modern methods for attempting to fit the most appropriate predictive equa- 
tion to such data are referred to as methods of subset regression. Prac- 
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tical uses of such techniques in materials development have been described 
in Collins, Quigg, and Dreshfield (1968), Eckert and Serafini (1968), 
Sandrock and Holms (1969), Eiselstein (1971), and Filippi (1974). Such 
techniques have been made feasible by the advent of large scale computers. 

Many differing statistical decision procedures have been incorpor- 
ated into the computer programs that are used to pursue the objectives of 
subset regression. Some of them have been described by Draper and Smith 
(1966) and criteria for their optimality have been discussed by Hocking 
(1972) . 

Two reasons for the plethora of methods are: 

1. There is no unique and widely accepted criterion for judging 
which of two proposed statistical procedures is the better. 

2. If a unique criterion is adopted for use in measuring the good- 
ness of a subset regression procedure, the analytical problem becomes 
mathematically intractable if the object is to devise a procedure of wide 
applicability that will satisfy the criterion. This results in many spe- 
cial solutions of special cases. 

In the present investigation, the inhibitions of mathematical trac- 
tability were thrown off by using Monte Carlo simulations. The method of 
subset regression that was investigated was essentially the method of 
chain pooling described by Holms and Berrettoni (1969) . Whereas the 
method had been investigated to minimize probabilities of Type 1 or 
Type 2 errors, the present investigation looks for strategies that mini- 
mize the maximum prediction errors. The results are offered as being 
generally useful because of the general applicability of the underlying 
assumptions. The assumptions are as follows: 

1. The model to be fitted is linear in the unknown parameters. 

2. The errors of the observations are independently normally distrib- 
uted random variables with a zero mean and a constant variance. 

3. Orthogonal estimators are available for estimating the unknown 
parameters of the linear model. (This orthogonality can be the result of 
the design of the experiment that furnished the observations, or it can 
be the result of an orthogonalizing transformation of the terms of the 
equation . ) 

4. The appropriate criterion of the goodness of a subset regression 
procedure is the smallness of the largest of the prediction error mean 
squares over the points of the experiment for which the experimentor de- 
sires to make predictions. 

5. There is no replication available for an estimate of the "pure 11 
error variance . 
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The first four of the five preceding assumptions are fundamental to the 
rationale of the method; however, data will be cited for believing that 
the method is robust against the normality part of the second assumption. 
The fifth assumption merely acknowledges the possibility that an alto— 
gether different method might be preferred in the presence of pure repli- 
cation. 

The original investigation of chain pooling had been concerned with 
three sizes of experiments, namely, experiments furnishing 16, 32, or 64 
observations (Holms and Berrettoni (1967)). The simulations had shown 
drastically reduced decision error probabilities, or the equivalent, 
greatly improved information efficiencies, for the larger experiments. 

Such results suggest that the method of analysis is relatively less crit- 
ical for the larger experiments, and the methods described (Holms and 
Berrettoni (1967)) are therefore believed to be adequate for producing 
small prediction errors with experiments providing 32 or more observa- 
tions . 

For relatively saturated experiments that are smaller than sixteen 
observations, the opinion is offered that such experiments are too small 
to provide both (1) good estimates of model coefficients and (2) a good 
test statistic, in cases where random errors are large enough to call for 
a statistical decision procedure. In other words, experiments with less 
than sixteen observations should be fitted with models of a fixed size 
with no use of conditional modeling. 

Consistent with the preceding remarks, the simulations of the pres- 
ent investigation were all performed with experiments containing sixteen 
observations in the belief that such experiments are large enough to 
justify the use of a statistical decision procedure, but small enough 
so that the precise optimization of the decision procedure would be quite 
beneficial. 

Where m p is the number of estimable terms deleted before the deci- 
sion procedure begins (or equivalently, the residual degrees of freedom), 
where a p is the nominal size of the preliminary tests, and where ctf 
is the nominal size of the final tests, the parameters of the chain pool- 
ing strategy are the quantities mp, a p , and a^. Results of the inves- 
tigation are presented as recommended values of m p , a p , and oif. 

The results thus provide an improved method for the mathematical 
modeling of the small, expensive, hard to control experiments that are 
typical of empirical research and development in materials optimization. 
The improvement lies in the fact that the statistical decision procedure 
is optimized to minimize prediction errors, thus maximizing the accuracy 
with which optimum materials compositions and processes can be identified. 
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EXISTING THEORY OF SUBSET REGRESSION 
Optimality Criteria 

The underlying population model for the system being investigated 
is assumed to be of unknown form and to contain an unknwon number of 
population parameters. What is known is that there is a set of variables 
, u„ whose values can be measured or controlled within a negli- 
gible amount of error. Another variable, called the dependent variable 
is assumed to be observable and to contain a random component called 
error where the error is independently normally distributed with mean 
zero and constant variance. In an abstract form, the model equation for 
the dependent variable is written 

Y = y(u 15 . • u ) + e (1) 

where Y is the dependent variable , uj_ , . . . , Ug are the independent 
variables , y is the unknown function, and e is the error. 

The next step of the procedure is to approximate the unknown func- 
tion by a polynomial in the independent variables, which will be linear 
in a set of parameters to be estimated. The parameters will be estimated 
by fitting some version of the polynomial to a collection of data con- 
sisting of t matched sets of observations of y and u;l, . . ., Ug. 
Whether or not the data was collected from any controlled activity, the 
data set will be called an experiment , and the sets of values of 
, . . . , Ug will be called treatments . 

In the absence of pathological cases, the availability of the t 
matched sets of observations allows the fitting of a polynomial contain- 
ing t unknown parameters. Thus, conceptually, an approximation to (1) 
is written 

Y — * * * * u ) • "b (u^, . • . , u^) ® (3) 

The assumption is now introduced that through prior knowledge, or 
through knowledge of the structure of the experiment, the functions f^ 
(u^> . . ., Ug) are all established. With these fi (uj_, . . ., Ug) 
being known functions, let 


x i 


= f ± (u 3 


V 


(3) 


and let 


x 


1 


- 1 


Equation (2) is now written 


* + 


Y t x t 


+ e 


Y = Yi + Y 2 x 2 + • • 


(4) 
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In the preceding equations, the coordinates , . . . , u g are 
properly called independent variables , but because the x^ are functions 
of the u lf .... u„, the x^ are not mathematically independent of 
each other. Their special role is often identified by calling them re- 
gressor variables, or simply, regressors . 

Suppose equation (4) is modified by retaining only r terms beyond 
the constant term (not necessarily the first r terms) . With 1 < r < t 
and with appropriate changes of subscripts, equation (4) is now written 

Y = + . . . + S r x r + e (5) 

The usual procedure is to find estimates of the by the method of 

least squares. Assume that this has been done and that the estimates are 
b^. The resulting predictive equation is 

Y = bg + b^x^ + . . . + b r x r (6) 

For any point u^, . . . , u g the associated Xi can be determined and 
the prediction error e p is 

e = Y - y (U x , . . . , u g ) (7) 

The mean squared prediction error can be expressed as the sum of a vari- 
ance and a squared bias. Where B('), E(’), and V(‘) are the bias, ex- 
pectation, and variance of (')> respectively, the mean squared error of 
prediction is: 

E(e p ) = E[Y - E(Y) + E(Y) - y] 2 

= E[Y - E(Y) ] 2 + 2E{ [Y - E(Y)][E(Y) - y] } + [E(Y) - y] 2 

- V(Y) + B 2 (Y) (8) 

As is well recognized, increasing r can usually be done to reduce 
B 2 (Y). As was pointed out explicitly by Walls and Weeks (1969) such in- 
creases in r are usually accompanied by increases in V(Y). Selecting 
r and the r terms to be retained is the central problem in subset re- 
gression; however, not all authors have chosen the minimization of E(e 2 ) 
as the objective of their procedures. 

Three popular methods of subset regression are known as (a) forward 
selection, (b) sequential deletion, and (c) stepwise regression. Some 
discussion of the three methods was given by Draper and Smith (1966). 
Many discussions of the relative merits of differing methods of subset 
regression have dealt with finding the best subset of terms for a fixed 
number of terms [Beale (1970)]. This seems to be an incomplete formula- 
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tion of the problem. The question should be: "What is the best subset 
of terms?" with no prior specification of the number of terms in the 
subset * 

Many of the subset regression procedures in common use do not en- 
vision the retention of a fixed number of terms , but instead allow the 
number retained to be determined by a statistical decision procedure. 

This point of view is incorporated into the procedures described by 
Efroymson in the publication by Ralston and Wilf (I960), by Sidik (1972a), 
by Holms and Berretoni (1969) , and by the works of many other authors as 
summarized by Draper and Smith (1966) . These procedures all use tests of 
significance, presumably in an attempt to control some combination of the 
probabilities of the Type 1 and Type 2 errors. 

If the model selection uses tests of significance applied to the co- 
efficients of a fitted equation, the question arises as to what form of a 
loss function is appropriate to the statistical decision procedure. The 
methods of classical statistics often give procedures whereby an arbitrary 
limit may be established to control the probability of Type 1 or Type 2 
errors. On the other hand, real life problems seldom provide the informa- 
tion for a rational quantitative specification of such particular proba- 
bility levels of decision errors. In many cases a reasonable objective 
is that of minimizing some function of the mean square error of predic- 
tion [as defined by eq. (8)]. In such cases a statistical decision pro- 
cedure might operate in the guise of significance tests, but its ultimate 
objective would be the minimizing of prediction errors, rather than the 
control or probabilities of Type 1 errors. 


Small Experiments 

The problem under consideration is that of fitting a model equation 
to an experiment that resulted in a small number of observations. Con- 
sistent with the smallness of the experiment, the assumption is also made 
that it contains no pure replication, so that t tests or other well 
established procedures are not obviously valid or applicable. 

Allen (1971) referred to literature that used the residual sum of 
squares as a criterion for choosing regressors* He pointed out that 
there are at least two objections to using such a criterion: 

1 * If the residual sum of squares were the only criterion, then all 
of the regressors would be used and there would be no motivation for sub- 
set regression. 

2. The residual sum of squares is not directly related to the "nat- 
ural" loss function which is the mean square error of prediction. 

On the other hand, the methods of Allen obtain a criterion called 
PRESS by predicting each observation from all the other observations* 

The present investigation is concerned with small saturated experiments 
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where each observation is more or less crucial to the estimation proce 
dure, particularly for predictions at the set of regressor values corre 
sponding to a given observation* The PRESS criterion is therefore e 
lieved to be inappropriate to subset regression for small saturated ex- 
periments* 


Nonlinear Models 

In the fitting of equations to data, a large body of knowledge has 
been developed for the fitting of linear models* The fitting of nonlinear 
models is a subject of current research outside the scope of the present 
investigation* (The initial use of a nonlinear model is presumably dic- 
tated by prior knowledge and such prior knowledge might dictate t e use 
of a different model selection procedure*) The restriction to linear 
models is no great handicap in fitting to data over sufficiently sma 
domains because nonlinearity of the response function can be approximated 
by fitting a linear combination of power functions of the independent 
variables* 


Colinearities 

The coefficients of equation (4) are usually estimated by the method 
of least squares which gives unbiased coefficient estimates* For any of 
several reasons, the data analyst may wish to set some of the coefficients 
equal to zero, or he may wish to adjust their values as a result of cer- 
tain deficiencies in the experiment. 

The existence of correlations or colinearities among the regressors 
constitutes a potential danger in the estimation of the parameters of 
equation (4) . Such a situation in combination with errors in the y-values 
can result in estimates of the y ± having excessively large absolute 
values. A direct treatment of this symptom is to use methods that produce 
biased estimates, as in the methods of RidRe A nalysis discussed by Hoerl 
and Kennard (1970) , and also by Marquardt (1970) . The methods of biased 
estimation have the obvious disadvantage that they provide no specific 
procedure for the discarding or selection of terms in equation (4). They 
have other disadvantages as illustrated by the simulations that were per- 
formed by Newhouse and Oman (1971) * Furthermore, when backward elimina- 
tion is used in subset regression, the experience is usually that each 
successive elimination results in smaller absolute values for the coeffi- 
cients that remain. This suggests that the objectives of these biased 
estimation procedures might also be attained by the use of a good subset 
regression procedure* 


Special Techniques 

Suppose that an experiment has been performed giving observations 
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of the dependent and associated independent variables , under the circum- 
stance that the error in y is independently normally distributed with 
mean zero and constant variance a ^ and that there are multicolineari- 
ties among the independent variables. One approach to such a problem is 
exemplified by the book by Daniel and Wood (1971) and consists of a mix- 
ture of much plotting and tentative model fitting with occasional uses of 
Mallow's (1973) C^-statistic all blended with generous infusions of the 
exper imentor f s prxor knowledge in an ad hoc manner for the particular set 
of data . 


CHAIN POOLING COUPLED WITH PRINCIPAL COMPONENTS REGRESSION 

The true form of the population model and the value of the error var- 
iance are assumed to be unknown. The best procedure is deemed to consist 
of estimating the parameters of a linear model containing as many param- 
eters as there are observations. An appropriate procedure is to be used 
to set some of the parameter estimates to zero. 

Two procedures available for setting parameter estimates to zero 
are Half-Normal Plotting (Daniel, 1939) and Chain Pooling (Holms and 
Berrettoni, 1969), Similarities and differences between Half-Normal 
Plotting and Chain Pooling will be discussed* Both of these methods re- 
quire single degree of freedom orthogonal estimates of model parameters* 
Such estimates are ordinarily provided by the results of two-level full 
and fractional factorial experiment^ without replication. The possi- 
bility of extending such methods to cases of correlated estimates through 
orthogonalizing transformations ("principal components regression”) will 
be discussed. 


Comparison of Half-Normal Plotting with Chain Pooling 

In Half-Normal Plotting, the absolute magnitudes of the coefficient 
estimates are plotted in an ordered manner on probability paper. All 
data interpretations are therefore dependent on the observed ordering, 
rather than on a prior ordering. Chain Pooling is similarly based on the 
observed ordering. It uses a sequence of dependent tests of hypotheses 
where each test is referred to the distribution function of the largest 
of a set of chi-square variates* 

Conditional model building is just one of the objectives of Half- 
Normal Plotting. Other objectives include the detection of nonuniform 
error variance and other departures from the usual assumptions. By com- 
parison , the objective of Chain Pooling is only that of conditional model 
selection, but it attempts to achieve its objective in a manner that has 
been established as being approximately optimal by combining a few 
rational considerations with a huge amount of Monte Carlo simulations. 



9 


Procedure Based on Prior Ordering 

Chain pooling uses a succession of tests of significance- Another 
procedure using a succession of tests of significance was proposed by 
Kennedy and Bancroft (1971) . In contrast to the methods of Half-Normal 
Plotting and Chain Pooling, they use a prior specified order for the 
tests of significance. 


Deletion Under the F-Test 

Subset regression procedures often proceed as follows : A multi- 
dimensional polynomial is fitted to the available data. If there are g 
independent variables and t observations, a polynomial, truncated at t 
terms is written having the form: 


Y - + 


; .A ..A 

^ 6.x. + ' ^ . ' e, d x.x, + 


g g 


/ 

i=l 


-i- i ••'/■/ ij i J 

r=i j=i 


B. m x.x.x. 
/__/ /__/ / ./ ijk i 3 k 

i=l j=l k=l 


+ e + . . . + similar terms up to degree d 


(9) 


Thus far, the assumption is that a polynomial with t terms is to 
be fitted to t observations. Assume instead that r terms beyond Bq 
are fitted, with r + 1 < t leaving 


m n = t - (r + 1) 
pO 


10 « 
pO 


> 0 


(10) 


degrees of freedom for the residual sum of squares . In this case mpQ 
will be called the initial degrees of freedom for error. 


Let S m pO be the initial residual sum of squares with m p o degrees 
of freedom. P Assuming that the residual sum of squares is due only to 
error (that is, assuming zero lack of fit), the quantity S^q/o 2 is a 
central chi-square variable with degrees of freedom. Assume that 

among the r terms in the initial model, one or more have zero popula- 
tion coefficients, and assume that one of these terms, namely the i*" 
one, is selected at random, is deleted from the model, and is pooled with 
the residual. Then the degrees of freedom for the residual is increased 


by one. Let the new residual sum of squares be 
ceding assumptions the ratio 


- > mp , i • 


Under the pre- 


F = 


(S 


m P. 


- W' 1 


S r,/m n 
mp , 0 pO 


(ID 


has the F-distrlbution with 1 and nipQ degrees of freedom. 

If only one test were to be performed, equation (11) could provide 
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an a-level test of the null hypothesis 3 ± = 0 against the alternative 
hypothesis ^ 4 0. The critical region would be the region for which 
the test statistic exceeds the (1 - ct) point of the cumulative distribu- 
tion function of F with 1 and ^pO degrees of freedom* Xf subsequent 
tests are to be performed, the question arises as to whether their sensi- 
tivity can be improved by increasing the degrees freedom of the denomi- 
nator through the pooling of mean squares into the denominator correspond- 
ing to coefficients that had previously tested as nonsignificant. Such a 
practice would invalidate the ct— level of the F-test. 


The Largest of a Set of Chi-Square Variates 

Assume now that j - 1 terms have been deleted from the model that 
originally contained t terms, and that Rj_]_ is the residual sum of 
squares corresponding to the j — 1 terms, and that all of these terms 
actually represented population parameters with zero values. Let 
Z-m be the decrease in the regression sum of squares resulting from de- 
leting the i c ^ term from the regression equation, and let Z^j be added 
to Rj i to form the new residual sum of squares. Assume that all of 
the deleted parameter estimates were independently chosen and orthogonal. 

If the parameter coefficient of the i*-h term were zero, the sum Z^j + Rj-l 
would then be the sum of j independent single degree of freedom central 
chi-square variates. Furthermore, assume that Z-y is the largest of 
these chi-square variates. Let 


C J - V ( Vi + V ( 

Then C.. has "Cochran’s" distribution as described by Cochran (1941). 
An upper critical point of Cj thus provides a critical region for a 
test of the hypothesis 


Multiplication of C. by j gives the Uj distribution tabulated by 
Holms and Berrettoni (1967) . 


Chain Pooling 

Consider a sequential deletion procedure that operates as follows: 
The residual sum of squares has mp degrees of freedom. A hypothesis 
test is performed against the Uj distribution where j - 1 = nip and 
Zj is the smallest, over all terms in the model, of the decrease, Z^j , 
in the regression sum of squares on deletion of the i^ 1 term. 

Form the statistic 


W. = jZ./(R. 1 + Z.) 
i J j y 


( 13 ) 
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Because of the violations of the assumptions of Cochran 1 s t distribution 
that are introduced by this procedure, Zj 4- Rj_i not ^ general be 

the sum of j random variables independently drawn from a single 
distribution with Zj being the largest among them. As a matter of 
fact , in typical regression problems , an arbitrary initial residual with 
nipQ degrees of freedom might be large enough so that with j - 1 = 

Vi /m po =• Z ] 

On the other hand, as the sequential deletion procedure continues, 
by increasing j one unit at a time with each ^j+1 — ^ j > the statistic 
Wj should become an increasingly better approximation to Uj, Accord- 
ingly, the sequential deletion procedure will use the upper 1 — tx per- 
cent points of the Uj distribution as the critical values of a decision 
procedure based on the statistic Wj . Furthermore, the sequential dele- 
tion procedure wiil use the conditional pooling of the chain pooling 
strategy of Holms and Berrettoni (1969) with components (mp0,<2p, and a f ) . 
In this strategy, m p Q is the number of degrees of freedom in the initial 
residual. The decision procedure then consists of sequential compound 
hypothesis testing. The procedure begins using the Wj statistic at 
significance level a . As long as Zj tests nonsignificant at level 
a it is pooled into the Rj-1 component of the denominator of Wj in 
preparation for the next test in the sequential deletion procedure. 

With the first Zj testing significant at level a p , testing at level 
a and the pooling of mean squares into the denominator ceases, and 
testing at level ctf begins. Suppose at this point that j = j p . At 
this point, Rj_i is no longer augmented with values of Zj that are not 
significant at level af. Furthermore, the critical value of Uj re- 
mains fixed at its value for j = j p and testing of the Zj continues 
until some Zj tests as significant at level a f . Suppose this occurs 
at jf. Then the associated regression coefficient is retained in the 
fitted equation and the decision procedure (sequential deletion) termi- 
nates. Within the strategy (nw)» a p> a f) » the consideration of a f > a p 
is unreasonable, and so the values of a p and a^, for which Uj has 
been tabulated and that are to be investigated, are as listed in table XI. 

The use of the Uj statistic is based on the assumption that the 
random observation error is independently normally distributed. As was 
shown by Holms and Berrettoni (1967) the operating characteristics of the 
chain pooling procedure remain essentially unaffected for an error with 
the same mean and variance but with a rectangular distribution. This re-, 
suit is not surprising in view of the fact that the coefficient estimates 
are always a sum of 28”^ observations, for which approximate normality 
can be expected, according to the central limit theorem. In other words 
the chain pooling is regarded as being generally robust with respect to 
the assumption of normality. 
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Principal Components Regression and Model Deletion 

The possibility of extending a chain-type decision procedure to cor- 
related estimates was suggested by Kennedy and Bancroft (1971) wherein 
they would apply or thogonalizing transformations to the original regres- 
sors. The use of orthogonalizing transformations in regression analysis 
had been suggested under the name of principal components regression by 
Kendall (1957) . Further discussion of principal components and subset 
regression was provided by Massy (1965) . The implications of combining 
principal components with chain pooling will be discussed in this section. 
Some insights into these matters are provided by numbers called the char- 
acteristic roots or the eigenvalues of the orthogonalizing trans formal 
tions. The roles they play will be more fully discussed in a subsequent 
section. It deals with the transformations back to the original regres- 
sors that follow after a subset regression has been performed on the 
orthogonalized regressors. 

In the method of principal components, the regressors of equation (A) 
are transformed to an orthogonal set and associated eigenvalues (charac- 
teristic roots) . . . , are also computed. Where the coordinates 

along the eigen vectors are w i , the fitted equation is 

Y = a Q + + a 2 w 2 + . . . + (14) 

The regression sum of squares is thereby partitioned into a set of mean 
squares (MSi) associated with the coefficient estimates and eigenvalues 
as follows (p. 70, Kendall (1957)): 

MS i = w.,a 2 (15) 

where is the characteristic root associated with W£ . 

In the procedure of Holms and Berrettoni (1969), where is the 

population coefficient of the i th term 

E(MS ± ) = a 2 + ta 2 

and if A-j_ is the noncentrality parameter for MS-^, then 

X i = ta i/° 2 ; E < MS i) = o 2 + A^ 2 = (1 + A ± )a 2 
and from equation (15) 


EOy* 2 ) = (1 + \ >o 2 


In his illustrative example, Kendall (1957) dealt with a situation 
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where two of the o)j_ were small and where the corresponding MS-^ were 
small. His discussion did not answer the question of what one would con 
elude or do if, for example, were small and w 5 were large, while 

MS 4 were large and MS 5 were small. An approach to this question was 
provided by Massy (1965) who differentiated between two circumstances as 
follows : 

"a. Delete the components that are relatively unimportant as pre- 
dictors of the original independent variables (X) in the problem; 
i.e., the components having the smallest eigen vectors should be 
dropped. " 


or 


M b , Delete the components that are relatively unimportant as pre- 
dictors of the dependent variable (Y) in the problem. In this case 
the components having the smallest values of gamma (the correlation 
between the components and Y) should be dropped." 

The objective of the present investigation is to minimize errors of 
prediction, correspondingly the point of view is not necessarily either 
one of Massy’s (1965) criteria "a" or "b." 

The predictive error of the equation resulting from a subset regres- 
sion procedure depends upon both bias error and variance error. The re- 
jection of too many terms causes excessive bias error, and the retention 
of too many terms can cause excessive variance error. The subset regres- 
sion procedure is to be optimized by minimizing the prediction error. 

But by the orthogonalizing process, the total variance of the observed 
dependent variable Is partitioned into a set of independent mean squares 
and the smaller of these mean squares represent contributions to the var- 
iance error while their associated terms in the orthogonalized regression 
equation do little to reduce the bias error. Thus a procedure such as 
chain pooling, which rejects terms corresponding to the smaller mean 
squares, could be ideal for providing a subset regression procedure that 
would minimize the predictive error. 

The basic concept of chain pooling is that beyond the a Q of (14), 
the coefficient estimates have been ordered in the decreasing order of 
their associated mean squares and that p of the estimates associated 
with the n smallest mean squares should be set equal to zero, while p 
of them are retained. Thus n and p are regarded as unknown integer 
parameters where 

p + n + 1 = t (16) 

The model equation is 


. . . + a w 

P P 


+ a 


p+l w p+l 


p+n w o+n 


Y = a Q + o 1 w 1 + 


+ ♦ . . + a 


(17) 
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The method of chain pooling concerned the hypothesis 

H • oc 1 1 a i o ► « • 1 cl i 0 
a p+1 p+2 p+n 

where n was to be estimated in some manner that would minimize some 
combination of weighted averages of the Type 1 and Type 2 error proba- 
bilities for the decisions = 0 . This optimal estimate of n was to 

be achieved using a decision strategy (mp , , otf) where the parameters 

of the strategy are as defined by Holms and Berrettoni (1969) * 

The point of view of the present investigation is that the basic 
chain pooling approach should be retained but that the components (mp* 
cap* ctf) of the strategy should be optimized for another criterion, namely 
to minimize the expected prediction error, that is , r = p should be 
chosen to give a prediction equation 

"" VN A 

Y = 4- a.w., + a 0 w n + . . . + a w (18) 

01122 rr 

/\ 

such that E(Y - u( W) ) 2 is minimized for some appropriately chosen value 
of W where W is a vector of values of the orthogonal regressors but 
not necessarily a vector of values for which the experiment was performed. 
Instead, W should be chosen according to the domain of intended applica- 
tion of the fitted model. 

When equation (18) has been transformed back into the coordinates of 
equation (4), the property of minimum prediction error achieved for Y 
is speculated to be retained. This speculation is based on a theorem 
that was proven by Kennedy and Bancroft (1971) for a decision procedure 
that has some similarities to chain pooling. Their analysis was con- 
cerned with a prior established order of significance testing and there 
is no direct connection between their distributional theory and the 
theory for chain pooling. What they showed was that the variance error 
and bias, and hence the mean square prediction error of their reduced 
models, is invariant under an orthogonal transformation of the regressor 
variables . 


Transformations Back to Original Regressors 

The subject of principal components analysis had been developed with 
a view to its application to situations in multivariate analysis where 
all the variables are regarded as random variables. Such a development 
was given by Kendall and Stuart (1966), Vol. 3, pages 285-289. The pres- 
ent discussion of regression analysis deals with situations where the in- 
dependent variables are observed or controlled with negligible error. 
Nevertheless the actual values given to the independent variables may be 
looked upon as random variates and thus for any given experiment, many 
aspects of the spatial distribution of the points of the regressor vari- 
ables may be studied from the point of view of principal components 
analysis . 
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A valuable set of numbers characterizing the spatial distribution of 
the points of the experiment is the set of eigenvalues associated with 
the transformation of the original regressors to an orthogonal set. The 
relative magnitudes of these eigenvalues are measures of how widely spread 
the experimental points are in the directions of the eigen vectors. Thus 
a severe multicolinearity in the original variables is a consequence of 
the spatial distribution of the points being concentrated near a multi- 
dimensional plane. One of the new coordinates will be normal to this 
plane and its associated eigenvalue will be very small (Kendall and 
Stuart (1966)). 

Some insight into the implications of the eigenvalues can be gained 
from figure 1, which compares two situations with differing eigenvalues. 
The true (but unknown) values of the dependent variable are assumed to be 
given (as a function of the regressor variables X]_ and X 2 ) by the 
dashed contour lines. A strong colinearity between xi and x 2 is 
illustrated by the distribution of the points of figure 1(a) which after 
transformation to principal components would be evidenced by a small 
value of o) 2 • Setting W£ = 0 imposes the subsidiary condition of 
X 1 = x 2 anc * ^is i s a mathematical statement of the fact that the space 
of the Z Xi was not well spanned by the experiment. The prediction equa- 
tion can be either 


Y 


= b, 


+ b 1 x 1 


or 


Y 


4- b -^x 


2 


but the uses of the equation must be restricted to values of x^ and x 2 
such that 


X 1 55 x 2 

In figure 1(b) , neither nor u)2 are small, however, the true popu- 

lation is such that MS 2 (and therefore a 2 ) is not significant ♦ Because 
of the insignificance, the predictive equation in the principal components 
is 


however, the predictive equation in the original coordinates contains 
both of them: 


Y = b Q + h i^ x i + x 2^ 

because the values of u)j_ and ^ have now shown that the space of the 
x-^ is well spanned by the experiment. Also, this fact and the insignifi- 
cance of MS 2 has shown that Y does not vary with W 2 , namely Y is a 
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constant on any line for which x^ + X 2 is a constant. The preceding 
discussion illustrates a typical usage of the combination of principal 
components analysis with subset regression analysis. One situation re- 
sulted in the two equivalent equations. 

Y = 6 0 + 


or 


Y Bq + 6 i x 2 


whereas the other situation resulted in the model 


Y 55 Bq + + Xj) 


where the values of the eigenvalues were crucial to the selection and 
usage of the final form of the model in the original regressors. 

The general situation now appears to be as follows* If the avail- 
able experiment has colinearities in the original regressor variables 
and does not furnish orthogonal estimates of the model parameters, the 
methods of principal components can be used to provide orthogonal regres- 
sors that will furnish uncorrelated estimates of the coefficients of the 
orthogonal regressors* This process will furnish a set of eigenvalues 
for the matrix product of the transpose and the design matrix of the 
original regressors. The relative magnitudes of these eigenvalues now 
present information as to how badly the original regressors are corre- 
lated, or equivalently as to how badly ill-conditioned is the matrix 
whose inversion would ordinarily be attempted. These eigenvalues are to 
be judged from the standpoint of numerical analysis or computer round-off 
errors or similar disciplines. The eigenvalues cannot be made the sub- 
ject of a statistical decision procedure, 

The elements which are made the subject of the chain pooling statis- 
tical decision procedure are the MS-j_ that are associated with the 
orthogonalized regressors. The deletions are therefore in the order of 
the smallness of the MS^ which from equation (15) are due either to a 
small (as illustrated by fig. 1(a)) or to a small (as illustrated 

by fig. 1(b)) or to both. The distinctions among these cases are vital 
to the construction of a final model in the original coordinates. A small 
value of MSj[ accompanied by a small value of a)*j_ implies that the co- 
ordinate should be set equal to zero and setting that eigen vector 

equal to zero imposes a single degree of freedom constraint on the space 
of the original regressors implying that predictions are valid only for 
combinations of the original regressors such that Wj_ = 0. On the other 
hand, a small MS-^ accompanied by a large would imply that is 

small and that the dependent variable is essentially constant along lines 
in the space of the original regressors for which all the orthogonalized 
regressors except are constants. In this case the term with coeffi- 
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cient a-j_ would be deleted from equation (14), and correspondingly, the 
original model would be limited to one less parameter, but it could con- 
tain all of the original regressors* 

The preceding discussion has been given in terms of linear trans- 
formations between the x^ and the w^. If the transformation between 
the ^ and the (eq* (3)) are also linear, the steps in the applica- 

tion of principal components regression are as already described* If on 
the other hand, the of equation (2) are nonlinear in the u-^ , the 

consequences for the u^ of setting some of the of equation (14) 

equal to zero would deserve special attention* 


CHOICE OF TRUE (POPULATION) MODEL FOR SIMULATIONS 
Simulation Type Investigations 

Optimality of model selection procedures is to be investigated* An 
analytical investigation could provide answers in general terras* Because 
of the complexity of the underlying statistical procedures, an analytical 
investigation will not be attempted* Instead, the investigation will be 
conducted by the performance of Monte Carlo simulations of experiments* 
Such empirical investigations have the risks that the conclusion might 
have severely restricted ranges of applicability. In discussions to fol- 
low, the rationale will be given for the choice of the designs of the ex- 
periments and for the choice of simulated population models so that the 
conclusions will have wide ranges of usefulness. 

The statistical decision procedure can be viewed as a game played 
between nature and the statistician. The statistician’s strategy is his 
model selection procedure and he seeks to minimize the prediction error. 
The strategy he should use is to be found by trial and error using com- 
puter simulations. Nature’s strategy must be built into the simulations* 
This can be done to represent nature as an indifferent opponent in which 
case the empirically optimized strategy will have a sort of Bayes opti- 
mality. The simulations could otherwise be done to represent nature as 
an aggressive opponent who would choose the worst possible population 
model against the statistician’s best possible procedure. Under these 
conditions the statistician’s empirically optimized procedure could be 
called a security strategy as defined by Luce and Raiffa (1957) * 


Unfavorable Population Models 

In an empirical study of decision procedures for model selection in 
the absence of pure replication (Holms and Berrettoni (1969)) the concern 
was not with prediction errors resulting from the choice of model equa- 
tion, but instead was confined to the count of decision errors in accept- 
ing or rejecting terms. That investigation of decision procedures was 
given a security quality by optimizing the recommended procedure against 
an ’’unfavorable distribution 11 of the parameters (regression coefficients) 



18 


On the other hand, if the procedure is to be optimized with respect 
to prediction errors in a manner that is also a security strategy , two 
"unfavorable" conditions should be introduced, namely, (1) the distribu- 
tion of the values of the parameters should he un favorable to the deci— 
sion procedure, and (2) the true shape of the function to be fitted 
should be unfavorable with respect to what would constitute a popular or 
reasonable form of the initial polynomial model. Thus the criterion ad- 
vanced here is not that the fitting and selection procedure should merely 
find good estimates of the true parameters of an unknown finite poly- 
nomial, but rather that the fitting and selection procedure should find a 
good predictive equation even if the unknown function is one that cannot 
be represented exactly by a finite polynomial. 

If the true shape of the function to be fitted is particularly un- 
favorable to the model selection procedure, then the empirically opti- 
mized procedure will have an approximate security optimality. But 
another type of optimality could be of interest. The simulations could 
be performed with the true shape of the function to be fitted being a 
type that is most likely to be encountered in practice. Then if the pro- 
cedure were empirically optimized to minimize mean square errors of pre- 
diction for such a function, the resulting procedure could be labeled as 
an approximate maximum likelihood optimal procedure. To keep the inves- 
tigation to a manageable size, and still keep it highly applicable to 
real life situations, conditions of the investigation will be chosen that 
will optimize the decision procedures against a blending of the two 
points of view just expressed, as follows: (1) To simulate the type of 

function expected to be met in practice (particularly in the practice of 
empirical optimum seeking) it will be an "inclined bell-shaped mountain" 
in several variables. Differing combinations of slope and "peakedness" 
can then represent the differing situations that might occur, for example 
in optimum seeking experiments that begin with the method of steepest 
ascents and then, depending on the observed topology, go to the method of 
1 oca 1 e xp 1 or at io n . To the extent that such a "bell-shaped mountain" is 
typical, this aspect of the investigation will result in decision proce- 
dures that have a maximum likelihood quality. The model that will be 
fitted to the results of an experiment will be a polynomial in several 
variables with a finite number of terms* The true function is unfavor- 
able with respect to such a model because the true model is a transcen- 
dental function, so that an exact polynomial representation would require 
an infinite series of terms. Of course, in any one simulation, a poly- 
nomial model containing t terms could be fitted exactly to the t ob- 
servations associated with t treatments, however, the simulation will 
add a random error component to the observed values of the function, so 
that the fitted polynomial will usually fail to predict the true function 
values at the coordinates of the treatments. (2) In such a situation 
(with the model containing a number of terms equal to the number of treat 
merits) some of the terms might be almost wholly error and thus add to the 
errors of prediction. Therefore a model selection procedure that rejects 
terms judged to be insignificant could operate to reduce the errors of 
prediction. The effectiveness of such a rejection procedure will be se- 
verely tested by giving the parameters values that, relative to each 
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other, are unfavorable to the rejection procedure. This will be done in a 
manner to be described under the heading of "Choice of Population Parameter 
Values Unfavorable to the Decision Procedure." 


Functional Form of Population Model 

The population model can be given the form of a bell-shaped mountain 
through an adaptation of the multivariate normal density function for g 
independent normal random variables. Such a function is given by 



where u k is the population mean of x k and is the standard devia- 

tion of x k . On any one coordinate, the function has a maximum at 
x k = y k and it has points of inflection at x^ = y k ± o k . The coordi- 
nates of the treatments consist of combinations of plus and minus ones. 

The population model will be made asymmetrical with respect to any plan of 
an experiment by locating its point of symmetry at one of the possible 
treatment points . 


The true (population) model will be a "multidimensional inclined 
bell-shaped mountain" because it will be given a functional form as 
follows : 


r 


iim 


QA1+ ; ^ k x lk + T m e 


k=l 




( 20 ) 


The parameters of equation (20) have purposes as follows: 

Hi£m dependent variable, function of the x^ k 

scale parameter. Because a = 1 the coefficient of variation is 
proportional to the reciprocal of 0^. 

coefficients of first degree terms 


t scale factor for exponential term 

m 
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coefficients of exponents 

6^ location parameters of mountain peak 

Other symbols of equation (20) are defined in appendix A. The plan 
of the experiment is a 1/2* 1 replicate of a full factorial experiment 
with g independent variables, and the number of treatments, t, is 
therefore 


t - 2 g ~ h (21) 

The subscripts have the domains 

i = 1, . . . , t 
k = 1, . . ., g 
i — 1, • ■ 
m “ 1 , • . « , ni^ 

Influence of Parameters on Shape of Surface 

As shown by equation (20) if x m = 0 the only odd degree terms are 
^l x il> • • •» ^gXig and therefore the surface defined by (20) is inclined 
with respect to the coordinates by the values given to the coefficients 
♦l» • • •, 4*g, and they define the approximate inclination of the surface 
with respect to axes through the origin if T m is small. These inclina- 
tions of the surface are fixed by the function 


A, 

i 



k-1 


Vik 


( 22 ) 


The bell shape is provided by the exponent of the exponential compo- 
nent of equation (20) and the exponent is 


C. 

i 


g 



\ (x ik 



(23) 


If x m is large in comparison with the values of the function given 

by equation (23) will provide a local maximum for (20) when (23) takes on 
the value of zero. The plan of the experiment will consist of treatments 
having coordinates such that x ifc — i-1* Also, the 5]^ will be given 
values of ±1. Thus the local maximum will occur at a corner point of 
the hypercube of the experiment, namely at the corner point for which 
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x ik " 6 k for k = 1, • • •» g* Furthermore, as is well known from the 
literature on the normal distribution density function, the function de- 
fined by equation (23) implies that the function defined by equation (20) 
will have two points of inflection on each coordinate axes. If these 
points are named x a k and they are located by the equations: 

x ak = 6 k " 1/(2 V 1 

(24) 

Xb, = S k + 1 /( 2^) 1/2 

The values of x a ^ for 6^ = 1 are given by table III. 

The magnitude of the function given by (20) relative to the error 
variance a* can be adjusted by adjusting the value of the population 
parameter 0^. Thus the effect on optimal estimation procedures of the 
relation between function mean values given by (20) and the error vari- 
ance a 2 can be investigated by assigning a fixed value to a 2 (namely 
- 1) and then investigating the effects of changes in the parameter 
(Thus the model has been made consistent with the usual assumption 
that is constant over the space of the experiment.) 


Choice of Population Parameter Values Unfavorable 
to the Decision Procedure 

The set of values (m p , a p , ctf) will be called the statisticians 
strategy . The set of values ^ , 0^, and i m (see eq. (20)) will be 

called nature's strategy. The question arises as to what constitutes a 
least favorable, or as an approximation, what will constitute a highly 
unfavorable set of values of the parameters in nature’s strategy? 

With respect to the decision procedure, and for the two level frac- 
tional factorial experiments, an unfavorable distribution of the param- 
eters (coefficients of the terms of the linear model) occurs if the param- 
eters (p in number) have the expectations of the order statistics of a 
normal distribution for a sample of size p. (See Holms and Berrettoni 
(1969).) Assume that observations are drawn with a sample of size g. 

Let the fractiles be qj where j = 1 The sample fractiles 

are 

qj = j/(l + g) (25) 

These values of qj for g = 4, 5, 6, 7, 8 are shown in table I. If 
the sample order statistics corresponding to these fractiles lie approxi- 
mately on a straight line on probability paper, then the sample is con- 
cluded to have been drawn from the distribution represented by the prob- 
ability paper . 
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Probability paper for the half normal distribution was used for fig- 
ure 2 and consists of coordinates for the fractiles , , and the fractile 

points, ip j . Straight lines were drawn on these graphs connecting the 
origin with the point defined by qj = qg and = 1. Values along 
these straight lines were then read to yield the values of shown in 

table II. Thus if these values of il>j are separate population param- 
eters for each of which single sample values are drawn, the sample values 
will approximate their mean values and therefore they will approximate 
the order statistics of samples of size g drawn from half normal dis- 
tributions. Thus the of equation (20) are likely to result in first 

degree regressor coefficient estimates of the fitted equation and associ- 
ated mean squares that will be declared insignificant by the decision 
procedure. Such decision errors have two undesirable consequences, 
namely, first some appropriate first degree terms are immediately lost 
from the fitted model and second, the denominator of the test statistic 
is made too large so that useful higher order coefficients are also de- 
leted. The distribution of the thus appears to be unfavorable even 

if the t m of equation (20) is nonzero. 

The t|)j therefore constitute a distribution of parameters unfavor- 
able to the decision procedure based on the Uj distribution and using 
the strategy (im,, ctp, af ) . (See Holms and Berrettoni (1969) for further 

discussion of this kind of an unfavorable distribution of parameters.) 

The particular values chosen for the ^ of equation (20) have been 
discussed in connection with the description of tables I and II. For 
simplicity, the values of the ^ of equation (20) will be set equal to 
the ^ values of table II. 

The complexity of the fitted equation needed to represent a popula- 
tion model depends on the surface irregularities that occur within the 
domain of the experiment. (A basic assumption is that good prediction . 
accuracy of the fitted equation is to be expected only within the domain 
of the experiment.) The domain of the experiment is defined by its 
corner points and they are located at x^ = ±1. Thus the domain is de- 
fined for all k by -1 = | 1, One type of surface irregularity that 

can occur within the domain of the experiment is a local maximum and as 
may be seen from equation (20) if r m is sufficiently large with respect 
to tj/fc, a local maximum occurs at x^ ** 6^ for any k. Another type of 
irregularity is defined by points of inflection. As developed in the 
discussion of equations (24) the coordinates of the points of inflection 
of the bell-shaped surface defined by equation (20) are given by equa- 
tions (24). Consider for example, the values listed in table III for 
g = 8. Then x ak of equation (24) is within the domain of the experi- 
ment for k = 1 , 2, 3, 4, 5, 6, and 7 but is outside for k = 8. (All of 
the points of inflection defined by x^k of equations (24) are outside 
of the domain of the experiment.) 

In view of the discussion of equation (19) the final form of the 
population model equation is an adjusted form of equation (20) namely 
it is 
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y i£m 


\ r fr'^M 2 

. . k-1 

Vik + T m e 


k=l 


(26) 


Eq lotion (27) results from substituting from equations (22) and (23) in 
equation (20) . With ^ ^ the useful computing equations are (22) , 
(23), and (27). 


±lm 


- V A i + v" c±) 


(27) 


Choice of Scale Parameter Values 

Nine values of 6^ will be investigated, namely, 0^ = 1/8, 1/4, 1/2, 
1, 2, 4, 8, 16, and 32. Three values of x m will be investigated, 
namely t m = 0.0, 1.0, and 2.0. The simulations will be performed for 
almost all combinations of these 0£ and T m values. 


EVALUATION OF DECISION PROCEDURE 


Following the selection of terms (where some of the coefficient esti- 
mates are set equal to zero) , the predicted values of the dependent vari- 
able can be efficiently computed for all of the possible combinations of 
the independent values, by the reverse Yates method of Duckworth (1965). 


Where egin are the "observation" errors, namely the pseudo normal 
random numbers generated in the n c h simulation, the "observations" are 
given by 


y 0i£mn = y i£m + e 0in 


1 , 2 , 


,g-h 


(28) 


After the model has been fitted and insignificant terms deleted, the 
difference between predicted values, ypij^mn, of the dependent variable 
for the n c ^ simulation and the population mean will be called the predic- 
tion error, and thus it is 


e pi£mn ^pifcmn y i£m 


i “ 1* 2 2& (29) 


Over the n e simulations, the sample mean square error of prediction 
for a given treatment is 
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_2 

e pi£.m 



n / / pi&mn 

n=l 


The maximum of such errors over the treatments is 


(30) 


Jl — 2 

e = e _ = max ie n . . 

max p£m,max \ pi^/ 

i-1, . . .,28 


The mean of the squared error over the simulations and over the 
points of the space of the experiment is 


e 2 = — e 2 

p£m 9 g pi£m 


Equations (31) and (32) provide two criteria for measuring the effec- 
tiveness of a strategy (nip, a p , otf) . The triad of particular values of 
in , a and a f that minimizes ^£ m>ipax (as given by eq . (31)) can be 
called a security strategy, and the triad of values that minimizes e"p£ m 
can be called an approximate Bayes strategy, if the points of the space 
of the experiment are assumed to be equally likely of being of interest. 

In either case, the absolute values of squared errors would have been the 
prime consideration. 

An example of a situation where such definitions of error would be 
appropriate occurs if the experimenter seeks to maximize some predicted 
response, such as the strength of a material as a function of its ingre- 
dients. For such an example, the region of the space of the experiment 
of greatest interest would be the region in the vicinity of the maximum 
point, where the function would most likely have its sharpest curvatures 
and largest errors due to lack of fit. As was discussed in connection 
with equation (23), this is the point where x-j^ = 6^. For SuC ^ an exam- 
ple, the appropriate criterion to be minimized for the choice of a strat- 
egy would seem to be the quantity eTpfctn^max equation (31). 

The criteria of equations (31) and (32) will be evaluated using com- 
puter simulations using (in most cases) 1000 experiments . Thus the long 
run mean squared error of the decision procedures will be evaluated. 

This leaves open the question of how badly a decision procedure might 
perform in individual cases. One approach to this question is to evalu- 
ate the stability of the mean squared errors observed in the simulations. 
Thus in addition to the criteria of equations (31) and (32) , two other 
criteria for the effectiveness of a strategy (nip, ct p , af) will be inves- 
tigated. They are concerned with the stability of the quantities defined 
by equations (31) and (32). The instability of these criteria can be 
measured by the variance of the square of the prediction error. If Y 
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is a random variable, the unbiased estimate of the variance of Y from a 
sample of size n e is given by 



(33) 


The random variate of interest is the squared error of prediction, namely 
e pi£mn* From equation (33) the estimate of the variance of e pij^ ran is 


V(e2 ) = 
v piJlmn 


n 


- 1 




\ X ( e 2 ) 

/ / '' e pi£mn' 

n=l 


/ 


n. 


2 . j jr 

n Au 


x 2 
e . 
pi£mn 




(34) 


Equation (34) gives an unbiased estimate of the variance of the squared 
error over n e simulations . The maximum of this quantity over the space 
of the simulated experiments is defined by 


V(e 2 ) = V(e 2 ) 

max fl.nr max 


max 
i-l , . 



2 

pi&mn 


) 


(35) 


The arithmetic mean of the variance of the squared error over the 
space of the experiments is defined by 


2« 




J 1_ 
2& 


V(e 


pi£mn 


) 


i=l 


(36) 


DESIGNS OF SIMULATED EXPERIMENTS 
Number of Treatments 

Although statistical decision procedures can be defined operationally 
for any size of sample, however small, the point of view of the present 
investigation is that an experiment should (1) furnish all of the informa- 
tion to estimate the model coefficients and (2) provide adequate test sta- 
tistics for the model selection procedure. A further assumption is that 
to provide reasonably good estimates for both the numerators and the de- 
nominators of the test statistics, the experiment must contain at least 
16 observations . 


Plans of Experiments 

An experiment plan will be defined as a sequence of t specified 
treatments involving g independent variables. The experiment will be 
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a l/2 h fraction of the full factorial experiment. The number of treat- 
ments is therefore as given by equation (21): 

t = 2§“ h 

I 

For the present investigation with g = 4, 5, 6, 7, and 8, and using 
16 treatments in each experiment, the plans (using the notation of Holms 
(1967)) will be generated one from the other as follows: 

g - 5, h = 1: 

X 5 " X 2 X 3 X 4 


X 0 = x 2 x 3 x 4 x 5 


g = 6, h = 2: 


X 6 " X 1 X 3 X 4 


X 0 = X 2 X 3 X 4 X 5 = X 1 X 3 X 4 X 6 


g - 7, h - 3: 


x 7 = x x x 2 x 4 


X,. = X 0 X 0 X,X, = X-X-X/X, * X,X 0 X,X, 
0 2345 1346 1247 


g = 8, h = 4: 


x 8 = x 1 x 2 x 3 


X 0 = X 2 X 3 X 4 X 5 = X 1 X 3 X 4 X 6 = X 1 X 2 X 4 X 7 = X 1 X 2 X 3 X 8 


These generators are also displayed by table IV. The corresponding 
groups of defining contrasts are displayed by table V. The plan matrix 
of treatments for the full factorial experiment on four independent vari- 
ables may be seen by looking at the first four columns of table VI. For 
the fractional replicates defined in table IV, the levels of the added 
independent variable for g = 5, 6, 7, 8 are shown by the fifth, sixth, 
seventh, and eighth columns of table VI, Yates 1 notation for treatments 
and their order is shown in table VII ^ The treatments of table VI (in 
Yates* notation and order) and some of the lower order elements of the 
aliased sets of parameters (for g = 4, 5, 6, 7, and 8) are shown in 
table VIII as derived from table V. 
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Aliased Parameters Initially Assumed to be Nonzero 

In the fitting of models to the fractional factorial experiments, 
one or more of three assumptions will often be made, in choosing param- 
eters for the initial model from aliased sets of parameters. 

1. A first degree term is always preferred to a two factor inter- 
action. 

2. A lower order interaction is always preferred to a relatively 
higher order interaction. 

3. The experimentor can label the independent variables X., . . .,X 
in the relative order of his belief as to their relative tendency to ® 
interact. (The design of blocked fractional factorial experiments ac- 
cording to prior beliefs about the parameters has been discussed by Sidik 
and Holms (1971) and also by Sidik (1972c) and (1973). 

Aliased sets of the lower order parameters corresponding to the esti- 
mates provided by the experiments are shown in table VIII. These esti- 
mates will be associated with, and given the name of, only one of the 
parameters in each of the aliased sets, and all other parameters in an 
aliased set will be set equal to zero. The parameters to be retained 
for the initial model will be selected in a manner consistent with the 
three previously stated assumptions. 

In the aliased combinations, the lowest degree parameters will be 
assumed to be the only possible nonzero parameters in any aliased set. 
Where there is more than one parameter of lowest degree, the parameter 
with the lowest sum of its subscripts will be assumed to be the only 
nonzero parameter of the aliased set. If there is more than one param- 
eter of lowest degree with lowest sum of subscripts, that parameter will 
be assumed nonzero which has the lowest first subscript, or lowest sum 
of its first two subscripts, or lowest sum of its first three subscripts, 
and so forth. 

Based on the preceding rules (and before the deletion of terms under 
the statistical decision procedures) the assumed nonzero parameters of 
the model equations to be initially fitted to the observations are 
exhibited by the equations of table IX. 


Space for Which Predictions Will Be Evaluated 


In the case of a fractional factorial plan, the parameter estimates 
are based on the observations generated from population mean values ac- 
cording to equation (26) for the particular treatments (particular com- 
binations of +1 and -1 for the independent variables) as specified by the 
fractional factorial plan. The estimated parameters then give a predic- 
tion equation which should be usable at least for predicting values of 
the dependent variable corresponding to the treatments actually present 
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^ particular fractional factorial plan# However, the experimental 

space consisting of all possible combinations of +1 and —1 for the inde- 
pendent variables can be said to have been "spanned" by the treatments 
actually used in the fractional factorial plan. The question arises as 
to whether the predicting equation should also be used for predicting 
values of the dependent variable for points in the space of the experi- 
ment that are not in the fractional factorial plan. The object in laying 
down the procedures of the present investigation is to try to simulate 
those assumptions that would ordinarily be made by an experiroentor . In 
^his case the question may be rephrased as: Does the making of predic 

tions at points which are points of the full factorial design, but not 

of the fractional design actually run, constitute an act of extrap - 
olation (not generally allowable) or an act of interpolation (generally 
allowable) ?" The assumption now made is that such predictions will be 
viewed as interpolations - that the fitted model ought to be useful in 
jnakXng predictions at all of the points within the space spanned by the 
experiment* Thus the equations of table XX will be used for making pre- 
dictions at all possible combinations of values of +1 and -1 for the X^- 
and not just those values listed in table VI* 

Correspondingly , although the fractional factorial plans all have 
2 g“h - 15 ^ the number of points for which predicted values will be com- 
pared with population values is always 28. Therefore for g = 4, 5, 6, 

7, and 8, the numbers of comparisons between population values and pre- 
dicted values will be 16, 32, 64, 128, and 256, respectively. For these 
comparisons, the predicted values can be obtained using the Reversed 
Yates Method’ 1 as described by Duckworth (1965) . 


COMPUTER PROGRAMS 
Flow Diagram of Program POOLES 

The main logic of the program POOLES is shown by the flow diagram 
of figure 3. The Yates method subroutine (section 9) as called in sec- 
tion 2, the computation and ordering of mean squares in section 3, and 
the sequential deletion procedure of section 4, are essentially the same 
as the similar operations of "POOLMS" (as given by Amling and Holms 
(1973)). ’’POOLES" differs from "POOLMS" mainly in sections 1, 2, 5, 6, 
and 7; the reason for the additional sections being that "POOLES" pro- 
vides a Monte Carlo evaluation of the pooling strategies based on errors 
of predictions# 


Details of Program POOLES 

Section 1: Declarations, constants , population means, and strategy * 

The constants defining the populations, the experiments, and the sequen- 
tial deletion strategy, are read from data cards in the following order, 
with the order of the fields being the same as the order of the symbols 
in the following description: 
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Format Description 

(13A6, A2) Arbitrary literal information such 

as particular use of program, 
date of last change, and so forth 


(3X5) 

LGMH , NE 

(15, 8F5.0) 

KG, (PSI(K), K=1 , KG) 

(15, 8F5.0) 

KG, (PHI(K) , K=l, KG) 

(15, 8F5.0) 

KG, (DEL(K) , K=l, KG) 

(15, 8F5.0) 

LTH, (THETA(L) , L=l, LTH) 

(15, 8F5.0) 

MTAU, (TAU(M), M=l, MTAU) 

(315) 

MP, KP, KF 


After the declarations and initial constants have been read, the 
major operation is the synthesis of the population mean values in accord- 
ance with equations (22), (23), and (27). The solutions of these equa- 
tions (the synthesized values of are to be stored in an array 

coded as YMU (I,L,M). The points in the space of the experiment are ac- 
counted for by the loop "DO 20 I = 1, IP." Within that loop, construc- 
tion of the and C-^ of equations (22) and (23) is done within the 

loop "DO 15 K=l, KG*" The loop contains some special NASA-Lewis func- 
tions available within the IBM 7044-7094 direct couple system. As used 
in POOLES the Boolean function AND ((1-1), J) gives the logical intersec- 
tion of two 36 bit integer arguments as a real function. As used in 
POOLES the Lewis shift function IARS (KM1 , AX) gives the integer function 
resulting from the accumulator right shift of the real variable AX by the 
integer number KM1 of binary places, 

The constructions of Pi£ m over the number of predicted values IP 

are then repeated for all values l « 1, . . . , and m - 1 

by the loops "DO 49 M=l, MTAU" and "DO 48 L=l, LTH. " The array YMU 
(I,L,M) then contains all the population means as determined by equa- 
tion (27). 

For the given experiment plan and for the established sets of popu- 
lation means, more than one model deletion strategy can be evaluated. 

The strategy to be evaluated, and its subsequent evaluation, begins with 
the statement "50 READ (5,824) MP, KP, KF. M On completion of the evalua- 
tion of a particular strategy, control is transferred back to statement 
50. Subsequent strategy investigations are initiated by reading addi- 
tional data cards according to statement 50. The operation of the pro- 
gram ends when such cards are exhausted. 

The error simulations are generated so that each strategy is com- 
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pared for the same set of n e time 2£“h random numbers. This is 
achieved by reinitializing the random number generator for each new 
strategy with the statement "CALL SAND (XS)." The procedure generates 
a sequence of pseudo random numbers with a rectangular distribution by 
taking the low order 36 bits of the product r r _i *K where r r _j_ = pre- 
vious random number and Tq = 1 and K “ 5^--*. This fixed point number 
is then floated and returned to the calling program as a floating point 
number between 0 and 1 (Tausky and Todd (1956)). 

The prediction errors and their squares are stored in the arrays 
ERSQ (I,L,M) AND ERSQSQ (I,L,M). These arrays are initially cleared by 
the loops terminating at statements 97, 98, and 99. The arrays ERSQ and 
ERSQSQ used a large amount of storage. Many simulations used L = 6 
and M = 6 but storage was exceeded with I = 256. To accommodate 
I = 256, the necessarily smaller values of L and M were used by mak- 
ing the appropriate changes in the DIMENSION and FORMAT statements. 

Section 2 : Simulations and model fitting , - The number of experi- 

ments simulated is NE. The performance of these experiments and their 
analysis is controlled by the loop: "DO 699 N=l, NE." Within each ex- 

periment, the pseudo-normal random numbers for the XT treatments are 
generated within the loop "DO 215 1=1, IT, 2," the transformation to 
approximate normality being that described by Box and Muller (1958) . 

Each set of IT random numbers for an experiment is used with all pos- 
sible combination of the population parameters 0^ and through the 

statements "DO 690 M=l, MIAU" and "DO 680 L=l, LTH. " For all of these 
cases, the simulated observation errors, as stored in RN(l), are to be 
added to the population mean values (stored in YMU (I,L,M)) for the par- 
ticular treatments (1=1,..., IT) that were specified for particular 
values of g by table VI. Accordingly as g=4,5,6, 7, or 8 (that 
is, according to the value of KG) control is transferred to statements 
204, 205, 206, 207, or 208. 

After synthesizing the "observed" values of YOBS (I) the "SUBROUTINE 
YATES" (section 9) ending with statement 909 is used to compute the 
array B(I) which contains (except for division by the number of treat- 
ments) the Yates estimates of the parameters of table VIII. (The opera- 
tion of the statements within SUBROUTINE YATES was described by Amling 
and Holms (1973).) 

Section 3: Construction and ordering of mean squares . - The mean 

squares are formed from the parameter estimates (for those terms beyond 
3q) and a pointer function is created within the loop "DO 309 1=1, IT." 
The strategy of doing no sequential deletion is represented by the code 
nip - 0, and if mp = 0, control is transferred to section 5. Otherwise, 
ordering of the mean squares is done within the loop "DO 313 J=l, ITM2." 

Section 4: Deletion of terms . - The statistical decision procedure 

using the strategy (m p , ctp, af) is applied in this section as was de- 
scribed for POOLMS by Amling and Holms (1973). The sequential deletion 
begins with the pooling of the initial residual within the loop 
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"DO 415 J=l, MP," The sequential deletion based on conditional pooling 
("chain pooling") then proceeds within the loop "DO 419 J-MPP1, ITMl." 
This results in the retention in the model of Bo and rejection from 
the model of all terms with mean squares as small or smaller than the 
ordered mean square with subscript JETA. 

Section 5: Predictions . - Predicted values of the dependent vari- 

able for all the 2& points of the experiment space spanned by the frac- 
tional factorial experiment are computed in this section using SUBROUTINE 
YATES and the reversed Yates method as proposed by Duckworth (1965) * 

The operation of Yates’ method followed by the "reversed Yates 
method" is illustrated by the following table for a 2^ experiment: 


YATES METHOD 


YOBS 


B 


B/FIP 



Y 1 + Y 2 Y 1 + Y 2 + Y 3 + Y 4 
Y 3 + Y 4 Y 2 ' Y 1 + Y 4 " Y 3 
Y 2 - Y 1 Y 3 + Y 4 " Y 1 - Y 2 

Y 4'V Y 4- Y 3- Y 2 +Y l 


(Y x + Y 2 + Y 3 + Y 4 ) /4 
(Y 2 - Y x + Y^ - Y 3 ) / 4 

(Y 3 + Y 4 - Y 1 - Y 2 )/4 
(y 4 _ y 3 - y 2 + yp/4 


REVERSED YATES METHOD 



YOBS 



B 

YPRED 

< Y 4 

- Y 3 - Y 2 

+ Y 1 )/4 

(Y 4 - Y 2 )/2 

Y 4 

Y 1 

(Y 3 

+ Y 4 ' Y 1 

- Y 2 )/4 

<Y 2 + Y 4 )/2 

Y 3 

Y 2 

(Y 2 

- Y 1 + Y 4 

- Y 3 )/4 

(Y 3 - Yp/2 

Y 2 

Y 3 

< Y 1 

+ Y 2 + Y 3 

+ Y 4 )/4 

(Y x + Y 3 )/2 

Y 1 

Y 4 


In the case of the computer program, there are 28~h parameters 
estimated from a fractional factorial experiment. They are exhibited 
by the prediction equations of table IX. The reversed Yates method re- 
quires the insertion of zeros for all parameters of a 2 g model that 
are not estimated in a 26“ h experiment. To make the 2 g predictions 
corresponding to experimental points of the full factorial experiment, 
the 2S - h coefficients together with 2 g - 2 g ~^ zeros are listed in 
the correct Yates order for the 2 g coefficients of a full factorial 
experiment. The statements of section 5 then combine estimates for such 
estimable parameters with zeros for the nonestiraable parameters in the 
reverse Yates method. This is done according as g = 4, 5, 6, 7, or 8 
by the transfer of control to statements 540, 550, 560, 570, or 580, 
respectively. This list is then reversed and the reversed Yates method 
is used as in section 5. 
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Section 6: Accumulation of errors . - The squared error for each 

prediction is accumulated (as required by equation (30) in the array 
ERSQ (I ,L ,M) as computed with the loop ”D0 609 1=1, IP." These accumula- 
tions are stored for each combination of L and M as indicated by the 
loops terminating at statements 680 and 690, and this process is repeated 
for each of the n e sets of IT random numbers as indicated by the loop 
terminating at statement 699 . For the purpose of computing the variance 
of the squared error of prediction, the quantity 



n=l 




(e 2 . I 
V pi£mn/ 


of equation (34) is computed within the loop ending at statement 609 and 
stored as ERSQSQ(I,L,M) . 

Section 7 : Determination of maximum and mean squared errors an d 

their variances . - The purpose of this section is to determine maximums 
and means of the prediction errors over the space of the experiment after 
the errors have been evaluated over that space by accumulating over the 
simulations. The accumulation over the number, n e , of simulations had 
been stored in the array ERSQ(I,L,M). For particular L, and M, the 
determination of the largest prediction error over the space of the ex- 
periment as defined by equation (31) is done through repeated use of the 
library subroutine AMAX1, which determines a real number as a function of 
two real arguments. This is done within the loop "DO 750 1=1, IP." The 
summation for the mean squared prediction error over the space of the ex- 
periment as required by equation (32) is also done within the same loop 
terminating at statement 750. After division by the appropriate divisors, 
these two evaluations of error are stored in the arrays (ERSQMX(L,M) and 
AVERSQ(L,M). The quantity 

n e 

P (e 2 .„ 

/ / \ pi£mn 

n=l 

is computed and stored as TEM within the loop ending at statement number 
750. The quantity defined by equation (35) is determined to 

be the maximum of the values of TEM as determined by 

E = AMAX1 (E ,TEM) 

A 

ry 

and from this maximum, V ( e £ m ) max is computed and stored with the state- 
ment 

VESQMX(L,M) = E/FNEM1 
The sum of the values of TEM as given by 
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F == F + TEM 

* —9 

is then used to compute V( e £ m ) according to equation (36) using the 
statement 

AVVESQ(L t M) = F/FEM1XP 

The computation ends if the data for Mp, KP, and KF is exhausted; 
otherwise a new strategy is investigated by returning control to state- 
ment 50. 


Details of Program MODEL 

The main work of the investigation was done with program POOLES . 

Some properties of the population model were explored using a second pro- 
gram called MODEL. A summary of the two programs showing their names, 
principal options and outputs is contained in table X. 

The program MODEL was used to exhibit the mean values of the popula- 
tion without any variability introduced by randomness. Thus population 
mean values were computed according to equation (26), but there was no 
random number generator to generate errors with specified properties as 
suggested by equation (28). The only errors are the round-off errors of 
the computing system, and the system consisted of an IBM 7044 monitor 
direct coupled to an IBM 7094 central computer . For that system, the 
round-off error typically occurs in the eighth significant figure of deci- 
mal output. The lack of intentional random error in the program MODEL 
was symbolized by showing - 0 in table X. 

The manner in which the components (mg, a pj a f ) influence the deci- 
sion procedure was described by Holms and Berrettoni (1969) . The manner 
in which these parameters are read into the computer program was described 
for a different program called POOLMS by Amling and Holms (1973) . A 
major distinction between POOLMS and the programs of the current investi- 
gation is that the value nu = 0 is now possible. Its use merely implies 
that no deletion of any kind will be applied to the initial model. There- 
fore, as suggested by the parameters of table X, the program MODEL gener- 
ates no random error, and uses no statistical decision procedures on the 
initial fitted models (as they were given in table IX), 

The output of MODEL results from the following: 

1. Population mean values are generated according to equation (26) 
and listed as YMU(I,L,M). 

2. The population mean values are used directly in the Yates proce- 
dure to estimate the parameters of the initial models of table IX. These 
estimates are listed as BILM(I,L,M). 
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3. The reversed Yates method (Duckworth (1963)) is used to compute 
predicted values for all of the 26 treatments of the full factorial ex- 
periment. Differences between these predicted values and the population 
values from equation (26) are defined as bias and are listed as the array 
named BIAS(I,L,M). 

Because 16 parameters are estimated from orthogonal contrasts of the 
16 population values for the 16 treatments of the experiment, the bias 
for these treatments consists merely of computer round-off error. For 
the cases of g > 4 (namely for the fractional replicate experiments) the 
bias values for the points of the full 26 experiment not included in 
the 16 points used for the estimating should be much larger than just 
round-off error. 


DESCRIPTION OF RESULTS 
Bias Errors 

The statistical decision procedure was investigated by fitting poly- 
nomial equations to simulations of fractional factorial experiments of 
type 26~ h where g = 4, 5, 6, 7, and 8, and where h = g - 4, The poly- 
nomials that might reasonably be fitted in such circumstances were listed 
in table IX. If such fitted models are to be used for making predictions 
at all of the hypercube points of the corresponding 26 , (full factorial) 
experiment, then bias errors can be expected for those predictions that 
are made at points of the space of the experiment that were not points 
for which observed values were recorded. In order to best understand the 
results of the Monte-Carlo simulations that will include both bias and 
variance errors, the errors due to bias alone should be investigated. 

As previously described, the program MODEL was written to exhibit 
the bias errors that exist when there are no random errors and when there 
is no model deletion. Some results are shown by table XII. As is shown 
by table XX, the case of g = 4 has a starting model with 16 coeffi- 
cients fitted to 16 ''observations 11 and with h = 0, predictions are made 

only for the points of the 16 observations. The result consists of 16 

predictions containing only round-off error and no bias error. Thus for 
g = 4 and h = 0 table XII shows the consequent zero bias for all com- 
binations of values of 0 and x as indicated by the fact that the max- 
imum absolute value of the bias is so listed for all combinations such 

that 0 - 1, 32, and x=0, 1, 2,4, 8, 16. 

In the cases of the fractional factorial experiments, the population 
model (as shown by eq. (26)) need contain no more than nine first degree 
terms for g = 5, 6, 7, and 8, for the case of x = 0. For this case, 
the fitting of the equations of table IX to the 16 Observations 11 from 
the fractional factorial experiments always results in zero bias as con- 
firmed by the results from program MODEL in table XII. 

For the cases of h > 0 (the fractional factorial experiments) and 
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for t ^ 1 (the presence of higher than first degree terms in the popula- 
tion model of equation (26)), the maximum absolute values of bias in 
table XII were always greater than zero* Furthermore the. bias values 
were always quite large in comparison with the largest population values 
for all values of t > 1, (The largest bias values were always less than 
20 percent of the largest population values for x - 1 but were always 
larger than 20 percent of the largest population values for t > 1 for 
the values of 0=1 and 0 = 32 as listed in table XII.) Based on 
these observations of very large bias with t > 1, the consideration of 
optimal decision procedure strategies (nip, a a f ) will be mainly limited 
to the conditions of t = 0 and i = 1. 


Strategy Options 

The conceptually simplest strategy is to do no deletion. Then no 
mean squares are initially available for a test statistic and this fact 
and the associated strategy will be symbolized by writing m p = 0. Con- 
sistent with such a strategy, there will be no mean squares tested for 
pooling which is represented by a p = 1.0 and none can be judged insig- 
nificant which is represented by ctf = 1.0, 

The choice of just the smallest mean square as the denominator of 
the test statistic is represented by nip = 1, and subsequent pooling and 
significance tests are represented by a p = 1 and otf S 1 as discussed 
by Holms and Berrettoni (1969). 

Consider the situation where t treatments have been used to pro- 
duce t observations and there are therefore t degrees of freedom to 
be partitioned between single degree of freedom estimates of model param- 
eters and the degrees of freedom for the denominator of the test statis- 
tic at the start of the decision procedure. If the data analyst does not 
have prior information on the complexity required of the model, the wise 
strategy would seem to consist of starting with the maximum possible num- 
ber of degrees of freedom for the model and therefore with the minimum 
possible number of degrees of freedom for the test statistic, namely > 
use nip - 1. The use of a p < 1 will then permit the denominator of the 
test statistic to be augmented with additional degrees of freedom thus 
increasing its sensitivity. Based on this line of reasoning, no investi- 
gations with POOLES were run with nip > 1. 

The combinations of a p and ot £ available for investigation are 
listed in table XI. Not all of these combinations were investigated with 
n e = 1000 because some preliminary work with n e = 10 had suggested 
that for values of 0 > 1 a strategy becomes ineffective if both (1) a 
very large value of ct p is used to severely limit the number of degrees 
of freedom for the test statistic and if (2) this is followed with a very 
stringent final test of significance as called for by a very small value 
of cxf. Consistent with this concept, the values of ot£ were mainly 
limited to = a /10 with the exceptions occurring for 0^1. 
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Strategies for which a p and Off are both large will be called 
agressive strategies. They tend to retain most of the original terms in 
the model. Strategies for which a p and otf are both small will be 
called defensive strategies. They have strong tendencies to discard 
terms. The intermediate strategies where a p is large and oif is small 
will be called moderate strategies. They tend to pool a relatively small 
number of terms into the denominator of the test statistic and because of 
the ordering of the terms by absolute magnitudes, the denominator remains 
fairly small, and so the final test at level oif is a relatively sensi- 
tive test, and not as defensive as it would be if a p had been small for 
the same ot£. 


Optimal Strategies 


Illustrative output from POOLES is shown by appendix C. Similar out- 
puts for many strategies (m p , a p , a f ) were scanned to find the strategy* 

giving the smallest value of e^ ax (eq. (31)) for given values of g, 9, 
and x. Results are shown by table XIII together with associated values 
of V(e 2 ) max (eq. (35)). The resulting values of e 2 ax for large 6 
and t = 0 can be compared with expected values. If the terms in the 
population model are much larger than the variance error, and if in the 
population model (eq. (26)), t = 0, then the number of terms that should 
be retained in the population model is g + 1, With a 2 = 1 and with 
g + 1 terms estimated from 16 observations the total expectation of 
error variance for the prediction based on g + 1 independently esti- 
mated terms is 


E(e 2 ) = (g + 1)/16 


From the preceding equation and from table XIII, the values of E(e 2 ) and 
i 2 ax for 0 = 32, t = 0, for the several values of g were as follows: 

g (g + 1)/16 (0 = 32, 

T = 0) 


4 0.3125 

5 .375 

6 .4375 

7 .500 

8 .5625 


0.3447 

.4177 

.5038 

.5745 

.6793 


Some of the optimal strategies of table XIII for g > 4 are seen to 
be quite defensive for large x even at the larger values of 9, where 
the variance error is comparatively small. The question arises as to why 
a defensive strategy would be employed in the presence of large bias and 
relatively small variance error. The population model surface was fairly 
complex as suggested by equation (26) and a maximum point was located at 
Sjq = -1, 1, -1, -1, . . but that point was not one of the observation 
points of the fractional factorial experiments. The irregular shape of 
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the population model could cause relatively large magnitude coefficients 
of the higher order terms of the fitted models to be retained, which 
would bring predicted values close to the population values for those 
cases where predictions were being made at points of the design where 
"observations" had been recorded. For the points of the space correspond- 
ing to fractions of the experiment not performed, the predictions could 
contain very large errors due to such high order terms. For these pre- 
dictions, the errors are reduced by the deletion of such terms and this 
is why the optimal strategies of table XIII were highly defensive (used 
small values of a p and a f ) for h > 0 when both 6 was large and 
x was *1.0. 

Aside from questions arising from t > 1 two questions that might . 
be asked are: "Under what conditions are some of the (mp, a p , af) strat- 

egies optimal?" and "Is there some particular strategy (nip, a p , a f ) that 
is widely useful?" An attempt to exhibit an answer to these questions is 
given in table XIV. It lists a p and a f as row and column headings 
and lists the parameter values g and 6 as tabular entries. Results 
for the purely first degree population model (t = 0) are shown by 
table XIV(a) and results in the presence of a small amount of surface 
curvature (x = 1) are shown by table XIV(b). These results show that for 
x = 0 and 6 small (6 £ 4,0) the strategy (nip, a p , af) = (1, 0.75, 0.10) 
is a widely useful Strategy, but that for x = 0 and large 0, the opti- 
mal strategy most often used otp = af, with decreasing values of a p and 
af for increasing values of 0. In comparison with these results for 
x = 0, table XIV (b) showed that relatively larger values of a p were 
optimal for x = 1. (Large values of a p tend to inhibit the pooling 
of mean squares into the denominator of the test statistic, which was 
apparently beneficial when the bias risk was increased by going from 
x = 0.0 to x = 1.0) . 

The strategy (m p , a p , af) = (1, 0.75, 0.10) was suggested as a 
widely useful strategy, particularly for those values of 6 
(1.0 = 0 b 4.0) where the relative influence of the variance error would 
be moderate and for the value of x (x = 0.0) where the bias error would 
be absent. Other values of 0 and x (table XIV) had resulted in other 
strategies being optimal. These results raise the question of: "Just 

how fast does the mean square prediction error and its variance change 
when the values of a p and af are other than optimal?" 

Some parameter values of g and 9 requiring relatively extreme 
values of mp , a p , and af are designated by the footnoted values in 
table XIV. How Fast the mean square prediction error and its variance 
changes when the value of a p and af are other than optimal for these 
footnoted cases of g and 0 are exhibited by table XV. (Some values 
of g and 6 having less extreme values of a p and af as optimal 
strategies are also exhibited in table XV.) The cases exhibited in 
table XV are limited to the low bias cases of x = 1.0. The values of 
®max that identify optimal values of a p and af are indicated by 
solid line rectangles. For comparison, the values of e^^. associated 
with the widely optimal strategy of (mp, a p , af) = (1, 0.75, 0.10) are 



38 


identified by dashed line rectangles . One conclusion that is evidenced 
by table XV, is that the values of e^ax anc * always increase 

very slowly from the optimal values of cip 
sive strategies (larger values of a and 


and ct£ for the more agres- 
otf). On the other hand (see 


q n 

particularly tables XV(c) and (g)), e Sax anci V(e^) inax increase quite 
rapidly for strategies that are too defensive (values of a p and cif 
that are smaller than optimal) . 


These observations suggest that the use of a moderate strategy when 
an aggressive strategy is optimal is not too serious - the differences 
are mainly functions of variance error. On the other hand the use of a 
defensive strategy when it is not optimal (as in some cases of large bias) 
can be disastrous. 


The overall conclusions from the results exhibited by tables XIII, 
XIV, and XV seem to be that chain pooling with 16 orthogonal estimates of 
regression coefficients with no estimate of pure error should be done 
with the strategy of (nip , otp, ctf) = (1, 0.75, 0.10). Such a strategy is 
a moderate strategy tending to retain many of the terms, if many terms 
are needed. If, with such a strategy, many of the terms are rejected, 
the true situation may be concluded to be one where the underlying popu- 
lation response surface is relatively simple. In such a situation, the 
fitted equation may be used (1) for data smoothing (for computing pre- 
dicted values at points of the space that gave the observed values), 

(2) for interpolation (for computing predicted values at points interior 
to the space spanned by the experiment, and (3) for extrapolation (for 
computing predicted values at points exterior to the space of the experi- 
ment) . Of course - as is well known for extrapolations - even if the bias 
error is small, the variance error rises very rapidly with the distance 
of the predicted points from the space spanned by the observed points) . 


If on the other hand, the strategy (trip, otp, ctf) = (1, 0.75, 0.10) 
leads to the retention of a fairly complex model equation (one with sev- 
eral terms of higher than first degree) then the use of a more defensive 
strategy might result in an equation with less bias when used for inter- 
polation or extrapolation, but the prediction errors would be large in 
any case (as shown by table XIII for g > 4 and t > 1) . In these cir- 
cumstances, the reasonable conclusion would seem to be that the reduced 
model obtained with (m p , ctp, otf) = ( 1, 0.75, 0.10) is satisfactory for 
data smoothing at the treatment points, but that more experimenting must 
be done (such as the performance of the composite experiments of Box and 
Hunter (1957)) if any interpolation is to be done. 


CONCLUDING REMARKS 

Model fitting procedures were investigated for the types and sizes 
of experiments that are appropriate to empirical materials optimization 
studies. The two-level, fractional-factorial, fixed effects experiments 
are often appropriate to such purposes, and they provide the highly de- 
sirable orthogonal estimates of the model parameters. Chain pooling is 
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an appropriate method of parameter estimation and selection for such ex- 
periments. 

For the cases of nonorthogonal experiments, the subject of principal 
components regression was examined and was found to provide a suitable 
procedure for making the results of such experiments amenable to the 
methods of chain pooling. 

In empirical optimum seeking (as in materials optimization) the cri- 
terion for model parameter estimation and selection should be the minimi- 
zation of prediction error. In the Monte Carlo study performed to find 
minimum prediction error strategies when chain pooling methods are used 
for model selection, simulations were performed using population models 
intended to represent response surface applications. The results led to 
the following recommendations : 

1. Use the strategy (nip, a , a f ) = (1, 0.75, 0.10). 

2. If many terms are deleted - if only a few terms of higher than 
first degree are retained - use the reduced model for data smoothing, 
interpolation, and limited extrapolation. 

3. If only a few terms are deleted - if more than a few terms of 
higher than first degree are retained - use the reduced model for data 
smoothing only. Additional observations are needed for predictions at 
points other than those already observed. 
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APPENDIX A 
SYMBOLS 


FORTRAN 

Mathematical 

symbol 



I 

i 

Subscript on treatments 


IT 

t 

Number of treatments 


J 

3 

Subscript on ordered mean squares 


K 

k 

Subscript on independent variables 


KG 

g 

Number of independent variables 


L 

l 

Subscript on parameter 0 


LTH 

^0 

Number of values of 6 


M 

m 

Subscript on parameter t 


MTAU 

m T 

Number of values of t 


N 

n 

Subscript on experiments 


NE 

n e 

Number of experiments 


MP 

m P 

Number of mean squares pooled initially 


LGMH 

g-h 

Experiment contains 2^ treatments 


PHI 


Exponential parameters, k - 1, . . g 


FSI 


First degree parameters, k = 1, . . g 


THETA 

h 

Scale parameters, l = 1, . . „ , l 


TAU 

T m 

Exponential parameter , m = 1 , . . . , m T 


YMU 

^i&m 

Population mean for treatment, JiV* 1 value 

of 0^ and m th value of x m 


x ik 

Value of k t ' 1 independent variable at treat 

ment . For two level fractional factorial ex- 
periments; at its upper level, = 1; at 

its lower level, x^ = -1. 


a 

P 

Nominal significance level of preliminary 

test 
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FORTRAN 

name 

KP 

KF 


Mathematical 

symbol 

ct£ Nominal significance level of final test 

Column number of U. table associated with 
specified a p 

Column nurrfcer of Uj table associated with 
specified ct£ 
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APPENDIX B 
PROGRAM POOLES 
C 

t 1.* 05 CL AF AT IONS, CONSTANTS, POPULATION MEANS, AND STRATEGY 

C 

DIMENSION REMARK! 14 ) , ALPHAIIHi TB(64,10>, PH I ( 8 > , 

IP S I ( 8 ) , T H E T A ( 6 ) , TAU t 6> , Y MU I 128,6*6) * RN(64>, IN0I64), Z(64), 
2fRSQ( 128,6,6) , EP SO SQC 128 , 6, 6 ) , ERSQMX(6,6>» AVERSUl6,6>, 
3VESQMX(6,6>, AWSSQ(6,6), DEL<8) 

COMMON KK r Y08S(256>, 8(256) 

LATA I ALPHA! I) » i = I, I l» /O. 001 ,0.002 ,0.005 ,0.01,9. 02 5,0. 05,0*10, 
10.25,3 .50* 0.75, l. C/ 

DATA ( (TQI I,J),J=1,10>,I=1»24) /0*0 ,%9 , 3.0 ,0 .0 ,0.0 , 0 .0 ,0.0 , 0.0 , 0. 3 
1,0. 0,2 • 0000, 1.99999, 1. 99997, 1 . 99986 , 1.99917, 1. 99687, 1.9877, 1.923,1 
2. 706, 1.382, 2. 9976 , 2 . 9960 , 2.9904, 2. 9809,2.951 ,2.904 ,2 .8 06 , 2 • 52? , 2 . 0 
366, 1.683, 3. 97b, 3. 962 ,3. 92 5 ,3* 8 70 ,3.760 ,3. 62 5, 3. 412 ,2.949,2*395, 1.9 
46 1,4.887,4*845,4. 758,4*65,4.44,4.21,3*89 ,3.237 ,2*658,2*134,5.74,5. 
56 3, 5.46, 5» 31,4.99,4. 68,4.28,3*57 ,2.893,2*371 ,6.51,6*33,6. I 1,5.87 ,5 
6.46, 5. 09,4.61, 3.8 3, 3. 11,2. 54, 7.20,6.96,6.65,6. 35,5.88, 5.44, 4.91,4. 
70 6, 3 .2 9, 2.69, 7. 81, 7. 52, 7. 10, 6. 78, 6. 26, 5* 75 , 5. 17,4.27, 3* 4 5,2* 82, 8. 3 

84, 8.01,7.53,7. 17, 6. 59, 6. 03, 5. 41, 4. 45, 3. 60, 2. 95, 8. 82, 8. 44, 7.95, 7. 53 
9, 6.09, 6. 28, 5. 61,4.62 ,3. 74, 3.07 ,9.26,8. 84,8.33, 7. 87 ,7. 13,6.50,5.81 , 
A4. 77,3. 37, 3. 17, 9. 67, 9. 21, 8. 68, 8. 16, 7. 37, 6. 71, 5. 99, 4. 92, 3. 99, 3. 27, l 
80.05,9.55,6.95, 8.42, 7.59,6.91,6.15,5.05,4. 10,3.37, 10.40,9.86,9.20, 
C8. 66, 7.79, 7.07, 6.30,5.17,4.20,3.46,10.72,10. 14,9*43,8.83,7.96,7.23 
0,6.44, 5.29,4.30,3.55, 11*01, LO. 40, 9. 64, 9. 00, 8. 12,7. 36,6. 57, 5. 40, 4. 3 
59,3.63, 11. 28, 10* 64, 9. 84, 9. 17,8. 28, 7.52, 6. 69, 5. 50, 4. 48, 3* 70, 11. 53, l 
FO • 86, 10. 03, 9.34, 8. 43, 7.65, 6.81, 5.60, 4.56, 3* 77, 11* 76, 11* 07,10*22, 9. 
G5 1,8* 58, 7.78,6. 92 ,5*69,4.64, 3. 84 , 11.98, 11* 28, 10. 40, 9.67, 8. 72, 7 .90, 
H7. 03, 5. 78, 4. 7 1,3. 90, 12 .19, 11. 48, 10* 58, 9. 83, 8. 86, 8. 02, 7. 13, 5. 87, 4. 7 
18, 3«96jl2« 3 9, 11. 68,10. 76,9.99, 8* 99, 8.13,7. 23, 5* 95, 4* 85, 4. 02, 12* 58, 
J 11.87, 10.93, 10. 14,9. 12,8.24,7.33,6.03,4.92,4.08/ 

DATA { (tb( I, J) ,J= 1, 10 I ,1=2 5,47)/ 12. 76, 12.05, 11.10, 10.29,9.23,8.34, 
17. 42, 6. 11, 4.98, 4. 14, 12*93,12. 22, 11. 26, 10. 43, 9.34, 8*44, 7* 51, 6. 18, 5. 
20 4,4.19, 1309, 12. 38, U. 41 , 19.56,9.44,8.54,7. 60,6.25,5. 10,4.24, 13.2 
34, 12.5 3,11.55, LO. 68, 9. 54,8.63,7.68,6.32,5.16 ,4 .30 , 13. 39 , 12 . 68 , 11 .6 
48, 10.78,9.64, 8. 72 ,7. 76,6. 38,5.22,4.35, 13.53, 12.82, 11 *80, 10.88,9*74 
5, 8.81, 7*83,6.44,5.28, 4*40 , 13*67, 12.96, 11*91, 10.98,9.83,8.89,7.90,6 
6 . 50, 5. 33, 4. 45, 13. 80, 13. 09, 12. 01, 11. 07, 9. 91, 8. 97, 7. 97 ,6.56,5.38,4.5 
70, 13 .93, 13. 21, 12. 10, 11.16,9.99,9.04,8.04,6.62,5.43,4.54,14*05, 13.3 
82, 12* 19, 11.25, 10*07,9. 11 ,8.11 ,6. 68 ,5. 48, 4. 58 ,14.17 ,13.43,12*27 , 11. 
934,10. 15,9.18,8.17,6.74,5.53,4.62, 14-29, 13.53, 12*35, 11.43, 10.22,9. 
A2 5, 8.2 3, 6. 80, 5. 58,4. 66,14.41, 13.63,12.43,11. 51 , 10. 29 , 9. 31 , 8 .29 ,6 .8 
05,5.63 , 4.70, 14 . 53, 13.73, 12 * 51, 11. 59, 10. 36, 9 . 37 , 8. 35, 6 * 90, 5.67, 4. 74 
C, 14.64,13.82, 12.59,1 1.67, 10.43,9.43,8.41 ,6.95,5.71,4.78,14.75, 13.9 
D l, 12.6 7,11.75, 10. 50,9.49,8.46,6.99,5.75,4.82 ,14.85,14.00,12.75,11. 
L8 3, 10. 57, 9.55, 8.51, 7.03, 5.79, 4.06 » 19.95, 14.09, 12.8 3, 11.90, 10.64,9. 
F6 1,6. 5 6, 7. 07, 5. 83,4.90,15.05, 14. 17 ,12. 90 ,11. 97, 10.70,9.67 ,8. 6 1,7*1 
Gl, 5.87,4.94, 15. 15,14.25, 12.97, 12. 04,10.76,9.72 ,8.66, 7. 15,5.91 ,4.98 
H, 15.24, 14.33, 13.05, 12. 11 ,10.82,9.77,8.71,7.19,5.95 ,5.0 L, 15.33,14.4 
10, 13* 12, 12. 18, 10. 88,9.82, 8.76,7.23 ,5.99,5.04,15.42, 14.47, 13. 19,12. 
J25, 10.94, 9.87, 8.8 1, 7.27,6. 03, 5.07/ 

DATA! (TB(I,J),J®l,iO), 1=48,64) /15. 50, 14. 54 , 1 3. 26 , L2. 32 , 11 • 00, 9.92, 



18 • 85 * 7*31* 6.07, 5* 10,15.58, 14*60, 1 3 • 3^ t 12*38, 11 .q 6,9.97 ,8*89, 7*35,6 
2. 11,5. 13, 15.66, 14.66,13.38, 12 .44, 11. 11 ,10.02 #8. 93,7. 39 ,6.14,5. 16,1 
35. 73, 14*72, 13.44, 12*50, 11. 16, 10*07 ,0*97,7.43,6.17, 5*19,15*00, 14*79 
4, 13*50 , 12* 56, 11 *21, 10. 12,9.01 ,7.47 ,6.20,5.22,15*87 ,14. 85,13*56,12, 
56 2, 1 1. 26, 10*17,9*05, 7*51, 6*23, 5*25, 15*93, 14*91, 13*62, 12*68, 11*31,1 
60.21,9.09, 7.55,6.26,5. 28, 15*99,14.97,13.67,12.73,1 1.36,10.25,9.13, 
7 7*59*6*29, 5*31, 16*05, 15*03, 13 * 72, 12*78, 11*40, 10*29,9* 17,7*63, 6* 32, 
05.34* 16. 11* 15* 10* 13*77,12.83* 11.44* 10*33,9.21, 7. 67 ,6.35,5.37, 16*17 
9, 15. 16, 13. 82* 12. 8 8* l U 48, 10. 3 7,9. 25, 7. 70, 6. 30, 5. 40 ,16. 23, 15. 22, 13. 
A67, 12.93, 11.52* 10.41,9.2 9,7.7 3,6.41,5.43,16.29,15.28, 13.92, 12.9 7, 1 
bl * 56, 1 0.45 , 9. 33, 7 *76* 6.44, 5*46 ,16* 34, 15*34,13* 97 , 13*01 , 1 1*60, 10*49 
C, 9.3 7,7*79,6.47,5.48, 16*39, 15*40,14*92, 13*05 ,11*64 , 10.53,9*41 ,7*82 
0,6*50,5*50, 16.44, 15*46,14.06, 13*09,11.67,10.57 ,9.45,7*05,6*53,5.52 
£,0.0, 0*0, 0.0, 0.0, 0*0, 0.0, 0.0, 0.0, 0.0, 0.0/ 

F E AD ( 5 » 80 0 > ( PEMAFK I I ) ,1=1,14) 
hRIT r (6,801> (REMARK M >,1=1,14) 

Pf AO (5,824) LGMH, N c 

I T = 2**LGMH 

1 TM l = IT-1 

ITM2 - IT-2 

FIT* IT 

FNF* NE 

IF (NE.EQ.Q) FNE * 1.0 

FNEM1 * FNE - 1.0 

READ ( 5,825) KG , ( PS I < K > , K- L , KG ) 

WFITE (6,842) (PSI(K), K=1,KG> 

READ ( 5,625) KG , ( PH I ( K ) , K= 1 , KG ) 

TRITE (6,843) ( PH I ( K ) , K = 1,KG> 

READ ( 5,825) KG » ( DEL ( K) » K*1,KG) 

WRITE (6,844) (DEL(K), K=1,KG) 

P £A9( 5 ,825 ) LTH, (THETA(L), 1=1, LTH) 

PEA 0(5,825) MTAU, (TAU(M), M=1,MTAU> 

IP = 2**KG 
F IP * IP 
IPP1= IP + 1 
FPN8IP = FNE*F I P 
FEM1IP = FMEM1 * FIP 

0(0 49 M=1,MTAU 
DO 48 L-1,LTH 
DO 20 1*1, IP 

A* 1.0 
C = 0.0 
DO 15 K= 1 , KG 
KM 1= K-l 
J = 2**KMi 
AX = A NO ( ( 1-1) , J) 

XI = 2*IARS(KM1,AX)-1 
DEBUG I, K, XI 
A = A+PSKK >*XI 

C = C + PH1IK)*CX I-DEL(K) ) ** 2 
15 CONTINUE 

YMUI I , L , Ml = THETAd )*(A + TAU( M)*FXPI-C> > 

DEBUG I, L , M, YMU ( I ,L ,M) 

20 CONTINUE 
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48 c ONT INUE 

4 4 CONTINUE 

50 R r AD < 5,824 > MP ,KP,KF 
MPP1 = MP+1 
CALL SAND C XS > 

DO 99 M=1,MTAU 
DO 9 8 L=I » L T H 
DO 97 1 = 1, IP 

DPSO( I,L,M)= 0.0 
r R SU SQ ( I » L , M > = 0.0 

97 CONTINUE 

98 CONT INUE 

99 CONT INUE 

2.- SIMULATIONS AND MOOEL FITTING 

DC 699 N = 1»NF 

IF I NE . E Q » 0 ) GO TC 200 

DO 213 I = I » I T 

CALL R AND ( PN( I > > 

DEBUG t, PNU) 

2 1 3 CONT INUE 

DO 215 1 = 1 , IT, 2 
fs SQRT(-2.0*AIOG(RN( I > M 
0= 6.2831 8 53*PN ( I +D 
FNUI= E*COS ( D > 

DEBUG P N t I > 

FN( 1+1 >= E*$IN(D> 

DEBUG RN ( I + 1> 

215 CONTINUE 
GO TO 201 
20 0 DO 209 1 = l , IT 
F N ( ! > - 0.0 
2 09 CONTINUE 
201 DO 690 M= 1 , MT AU 
DO 680 L= 1 » L T H 
KK = LGMH 


IF (KG 

. C Q. 

4) 

GO 

TO 

2 04 

IF (KG 

•Eg. 

5 ) 

GO 

to ; 

205 

IF (KG 

»EQ . 

6 ) 

GO 

TO 

206 

IF (KG 

.EQ. 

1 ) 

GO 

TO 

207 

IF (KG 

.EQ. 

B) 

GO 

TO 

208 


204 DO 214 1=1, IT 

YOBS I I )= YMU< I , L, M ) +RN( I ) 
DEBUG YC8S (H 
214 CONTINUE 
CALL YATES 
GO TO 300 

205 YOBS ( 1) = YMU ( 1,L»M>+RN< 1) 

YG6S( 2) = YMU ( 2 » L » M > +RN ( 2) 
YOBS < 3> = YMU( 19 , L, M)+RN( 3) 
YOBS ( 4) = YMU< 20,L,M)+RN( 4» 
Y DBS ( 5> = Y MU ( 21 ,L,M1+RN( 5) 
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YObS< 6) = YMU ( 22»L*M)+RN{ 6> 
YOBS { 7) = YMU l 7 » L » M) +RN I 7) 
YOBS { 8) = YMU ( 8 * L » M ) +RN I 8) 
YOBS1 9 > = YMUI 25 ,L» M) +RNI 9) 
Y0BSI10) “ YMUI 26 ,L»M)+RNl 10> 
YOBS ( 1 1 ) = YMU ( 1 1 ,L» Ml +RNI 11 > 
Y0BSIL2) = YMU ( 12 »L » M ) +R N( 12) 
Y 0 8 S ( 131 = YMUI 13 ,L, M) +RNI 13) 
YOBSI 14) = YMU( 14,L»M> +RNI 14) 
YOBSI15) = YMUI 31 »L» M)+RN< 15) 
YOBS < 1 6 > = YMUI 32, L, M) +RN ( 16) 
DO 250 1=1, IT 
DEBUG YOBS I I ) 

250 CONTINUE 
CALL YATES 
GO TO 300 


YOBSI 

1) 

=s 

YMU I 1 

,L,M)+RNt 

1) 

YOBSI 

2) 

= 

YMU ( 34 

, L , M ) +R N I 

2) 

YOBS ( 

3) 


YMU 1 19 

,L,M)+RNI 

3) 

YOBSI 

4) 


YMU I 52 

» L , M > +RN I 

4) 

YOBSI 

5) 

= 

YMU I 53 

,L , M> +PNI 

5) 

YOBSI 

6) 


YMUI 22 

, L , M ) +RN I 

6) 

YOBSI 

7) 


YMU ( 39 

,L , M ) +RN ( 

7) 

YOBSI 

8) 

-= 

YMUI 8 

, L , M ) +R N I 

8) 

YOBSI 

9) 

= 

YMUI 57 

,L,M)+RN I 

9) 

YOBSI 10) 

= 

YMUI 26 

» L , M ) +R N I 10 ) 

YOBSI 11) 


YMU I 43 

i L , M) +RN I ID 

YOBSI 12) 

■= 

YMUI L2 

, L , M) +RN I 12) 

YOBSI 13) 

= 

YMUI 13 

,L,M)+RN< 13) 

YOBSI 14) 

:= 

YMUI 46 

, L » M ) +R N I 14) 

YOBSI 15) 

= 

YMIJI 31 

,L , M ) +RN I 15) 

YOBSI 16) 

= 

YMU I 64 

♦ L » M) +RN { 16) 


DO 260 1=1, IT 
DEBUG YOBS! I I 
260 CONTINUE 
CALL YATES 
GO TO 300 


207 YOBSI 

D 

3S 

YMUI 

1, L 

M ) + RN I 

1 ) 

YOBSI 

2) 

- 

YMUI 

98, L 

» M)*RN( 

2) 

YOBSI 

3 ) 

S 

YMUI 

83, L 

,M)+RN( 

3) 

YOBSI 

4 ) 

s 

YMUI 

52, L 

,M)+RNI 

4) 

YOBSI 

5) 

is 

YMUI 

53, L 

»M > + RN I 

5) 

YOBSI 

6) 

= 

YMUI 

86, L 

,M>*RN{ 

6 > 

YOBS I 

7) 

r= 

YMUI 103, L 

, M ) +PN ( 

7) 

YOBSI 

8) 


YMUI 

8, L 

,M)+PNI 

8 ) 

YOBSI 

9) 


YMUI 121, L 

,M)+RN( 

9) 

YOBSI 10 > 

- 

YMUI 

26, L 

,M)+RN( 10) 

YOBSI 11) 


YMUI 

43, L 

,M)+RN( 11 ) 

YOBSI 12) 


YMUI 

76, L 

,M)+RN( 12) 

YOBS 1 1 3) 

= 

YMUI 

77, L 

, M ) +RN I 13 » 

YOBSI 14) 

ss 

YMUI 

46, L 

,M)+RNI 14) 

YOBSI 15) 

- 

YMUI 

31, L 

,M)+RN( 15) 

YOBSI 16) 

■= 

YMUI 128, L 

, M ) +RN ( 16) 

DO 270 I 

= 1 

,IT 
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DEBUG YQBStn 
2 70 CONT INUE 

CALL YATES 
GO TO 300 


20b YQBSI 

1) 


YMUl 1 , L 

,M) + RNl 

1 ) 

YOBS ( 

2) 

= 

YMUl 226, L 

, M ) «- RN I 

2) 

YOBS 1 

3) 


YMUl 211, L 

, M ) +RN l 

3) 

YOBSl 

4> 

= 

YMUl 52, L 

, M ) + RN ( 

4) 

YOBS l 

5 ) 


YMUl 181, L 

, M > + RN ( 

5) 

YOBSl 

6) 


YMUl 86, L 

, M ) +-RN l 

6) 

YOBSl 

7 > 


YMUl 103, L 

, M } + F N ( 

7) 

YOBSl 

3) 

= 

YMUl 1 36, L 

, M ) + RN < 

8) 

YOBSl 

9) 

_ 

YMUl 121, L 

t M ) +RN l 

9) 

YOBSl 13) 

- 

YMUl 154, L 

,M)*RN( 10) 

YOBSl 11) 

= 

YMUl 171, L 

> M >+RN( 11) 

YOBS112) 

* 

YMUl 76, L 

, M ) +RN l 12 ) 

YOBSl 13) 

3 

YMUl 205, L 

, M ) +RN I 13) 

YOBSl 14) 

= 

YMUl 46, L 

,M)+RN( L4> 

YOBSl 15) 

- 

YMUl 31, L 

,M)+RN( 15 ) 

YOBSl 16) 


YMUl 256, L 

, M ) +RN ( 16) 

DO 280 

i 

*lf 

IT 




DEBUG YOBS(I) 


280 CONTINUE 
CALL YATES 
GO To 300 

3.- CONSTRUCTION AND ORDERING CF MEAN SQUARES 

300 DO 309 1= 1, IT 

IND ( I > - I 

Z ( I ) - I+1>/FIT 

B(I> = Bin / FIT 
DEBUG B( I > 

309 CONTINUE 

IF (MP.EQ.OI GO TO 500 
C 

DO 313 J= l, I TM2 
TEST- Zl ITMl) 

IN* ITM1 

DO 312 NA=J, 1TM2 
IF(TEST-ZINA)) 312,312,311 

311 TEST = ZINA) 

I N= NA 

312 CONTINUE 

I T E M= IMDI IN) 

TEM= Z < IN) 

IND(IN)* IND(J) 

Z( IN> = Zl J) 

I NDI J ) = ITEM 
ZIJ) = TEM 

DEBUG INDI J),Z( J) ,Bl ITEM) 

313 CONT INUE 

DEBUG IND I 15) , Z ( 15) , B( 15) 

C 

C 4.- DELETION OF TERMS 
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JN= MPP1 
TEM= 0.0 
DO 415 J=1,MP 
INOX = [NDlJH-1 
B ( INOX > - 0.0 
T[y= tcm + H J ) 

DEBUG JN, INDX, R( INDX > , TEN 

415 CONTINUE 

TF (KF.FQ.m GO TO 500 
DO 410 J = MPP 1 , ITM 1 
FJN = ON 

TEST = FJN*Z( J)/(TEM+Z(J>> 

IF (KP.EO. il> GO TO 417 

IF ( TEST-T6( JN,KP > ) 416,416,417 

416 TEM •= TEM + Z ( J > 

INDX = INOlJI+1 
B ( INDX ) = 0.0 

JN = JN+i 

DEBUG JN, INDX, B(fNDX), TEM 
GO TO 419 

417 IF(TESI - TB(JN,KF>> 418,418,420 

418 INDX - IND { J ) + 1 
B( INDX ) * 0.0 

DEBUG INDX, B(INDXJ,TFM 

419 CONTINUE 
JETA * I T M 1 
GO TO 500 

420 JETA = J-i 

00 421 J = J E TA , IT 
I NOX = IND( J >♦! 

DEBUG INOX, B ( INDX) 

421 CONTINUE 

5.- PREDICTIONS 
500 KK = KG 


DEBUG 

JETA 





IF 

(KG 

• EQ* 

4) 

GO 

TO 

540 

I F 

(KG 

*EQ • 

5) 

GO 

TO 

550 

IF 

( KG 

. E Q . 

6) 

GO 

TO 

560 

IF 

( KG 

• EG* 

7) 

GO 

TO 

570 

1 F 

( KG 

.EG. 

8) 

GO 

TO 

580 

DO 

546 

1 = 1, 

IP 





IPP1MI = IPPl-l 
YOBS ( I > = B( IPP1MI) 
DEBUG YOBS < l > 

546 CONTINUE 
CALL YATES 
GO tq 600 

550 DO 551 1=1,20 

551 YOBS ( I ) = 0.0 
YOBS ( 13) = B! 14) 
YOBS (14) = B ( 1 3 ) 
YOBS ( 1 5 > = B( 16) 
YOBS (16) = 8(15) 



DO 552 1=21,32 
IPP1MI = 33-1 
552 YOBS ( I > » BI IPP1MI) 
DO 557 I- 1» IP 
DEBUG YOBSI 1> 

557 CONTINUE 
CALL YATES 
GO TO 600 

560 DO 56 1 1-1 , 52 

561 YOBSI I ) = 0*0 

Y 0 B S ( 3. 1 > = B ( 1 3 » 

Y 0 B S I 3 2 ) = BI14) 
YOBS ( 471 = B( 16) 

Y 08S ( 46 > * B ( 1 5 > 

DO 562 1=53,64 
1PP1M1 = 65-1 

562 YOBSI I ) - B I IPPiM [) 
DO 567 1 = 1, IP 
DEBUG YOBSI I) 

567 CONTINUE 
CALL YATES 
GO TO 600 

570 CO 571 1=1,117 

571 YOBSI I > » 0.0 

YOBS I 64) = B< 12 ) 
YOBS I 9 5) = B ( 1 3 > 

YOBS I 9 6) = BI 14 > 
YOeSI 111)= B ( 16) 
YOBSI 112)= BI 15) 

DO 572 1=118,128 
IPPIM! = 129-1 

572 YOBSI I ) = 61 IPPLM I) 
00 577 1 = 1, IP 
OEBUG YOBS ID 

577 CONTINUE 
CALL YATES 
GO TO 600 

580 DO 581 1=1,245 

581 YOBSI I > = 0-0 
YOBSI 128) = BI 8 ) 
YOBSI 192) = BI 12) 
YOBSI 223) = BI 13) 
YOBSI 224) = BI 14) 
YOBS I 2 39 > = B ( 16) 
YOBS I 240 ) = BI 15) 

DO 582 1=246,256 
IPP1MI = 257-1 

582 YOBSI I ) = BI IPPIM I ) 
YOBS I 249) = 0.0 

DO 587 1=1, IP 
DEBUG YOBSI I ) 

587 CONTINUE 
CALL YATES 
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GQ Tq 600 
C 

c 6.- ACCUMULATION OF ERRORS 

C 

600 DO 609 I = 1»IP 
IPPLMI = IPP1-I 
DEBUG S( IPP1MI » 

TEM= ( B( I PP LM I ) “ YMU( >*'*2 

ERSQ(I»L*M) = EPSQU,L,M> + TEN 1 
ERSOSQ( l,L,MI= ERSUSQC + TEM**2 
DEBUG M,L» I,ERSQ< I*L,M> 

609 CONTINUE 
680 CONTINUE 
690 CONTINUE 

IF < NE * EQ » 0) GO TO 700 
699 CONTINUE 

7.- DETERMINATION OF- MAXIMUM AND MEAN SQUARED ERRORS 


700 DO 790 M=i»MTAU 
00 780 L=1,LTH 
C = 0.0 
0 = 0.0 
E* 0.0 
f= 0.0 

DO 750 I=itIP 
C = AM AX 1 ( C » ER SQ( I,L,M)) 

D = D + ER SGI 1,L,M> 

T EM= ERSQSQI I »t,M>- ( ( ER SQ ( I » l »M > > **2 ) /FNE 
E = AMAXI(E»TEM> 

F = F + TEM 
750 CONTINUE 

E R SQMX ( L » M ) = C/FNE 
AVEPSQ(LtM) = D/FENBIP 
IF (NE .EQ. 0) GO TO 780 
VE SQMX ( L i M > = E/FNEMI 
AWE$Q(L,M> = F/FEM1IP 

DEBUG ERSQMX(L,M> , AVER SQ ( L , M )» VESQMX ( L » M )» AWE SQ ((-» M 1 
780 CONTINUE 
790 CONTINUE 

C 8.- OUTPUT 

C 

WRITE (6,830) IT, NE,KG,MP,ALPHA(KP),ALPHA(KF > 

WRITE (6,832) ( TA UI M ) , M= l , MTAU ) 

WRITE (6,833) 

WRITE (6,831) 

WRITE (6,834) (THETAd ) , (ERSQMX(L, M) ,M=l,MTAU) ,L=l ,LTH» 

WRITE (6,835) 

WRITE (6,834) ( THETA ( L >,< AVER SQ(L , K ), M= 1 , MTAU) ,L*1 ,LTH > 
WRITE (6,836) 

WRITE (6, 834) ( THETA ( L >,( VESQM X ( L » M > ,M= i , MTAU) ,L=1 ,ITH ) 
WRITE( 6,837) 

WRITE (6,834) (THETA(L) ,< AWE SQ (L , M ), F= 1 , MTAU > , 1=1, LTH) 
GO TO 50 
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830 FORMAT ( 13 A6» A 2 > 

801 FORMAT <lHltlOX,12A6 f A2//> 

82 4 FOP >1AT (3 15) 

825 FOpmAT ( 1 5 i 8F5 • 0 > 

830 FORMAT 11H1,3X,4HIT =I5,5X*4HNE =I5»5X,4HKG =I5,6X,4HMP = 
18 HAL PH AP =F6 • 3 ♦ 5X 1 8 H AL PHAF *F6.3//> 

831 FORMAT ( 1HQ,2QX ,6HERSQMX/ / ) 

832 FORMAT C 1H0, 4HTAU-6F 13.2// > 

833 FORMAT ( 1H0 , 5HTHETA/ / ) 

834 FORMAT J F6 . 2 » 6E 14 .4 > 

835 FORMAT (1H0,20X,6HAVERS0//) 

836 FORMAT ( 1H0 , 20X , 6HVE SQMX / / > 

637 FORMAT ( 1H0 » 2 OX »6HAWE SO / / ) 

842 FORMAT ( 1H0 » 4HP S I ~8E 14.4 ) 

843 FORMAT ( 1H 3 , 4HPH I =8E 14 .4 1 

844 FOPMA T ( 1H0,4HDEL=3E 14.4) 

END 


5 *5X * 
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SUBROUTINE YATES 
C 

c 9.- YATES METHOD SUBROUTINE 

C 

COMMON KK , Y I 256), B( 256) 

I I = 2**KK 

I IDB2 * I 1/2 

KKM 1 = KK- 1 

DO 908 K= 1 t KKM 1 

DO 906 1 = 1 , II, 2 

! P 102 = II + D/2 

B( IP ID 2 1 = YU + 1> +Y( 1) 

LL - IPID2+IIDB2 

906 BILL) = Y I I ♦ 1 > — Y ( I) 

DO 907 I=l t II 

907 Y < I > = BID 

908 CONTINUE 

DO 909 1=1, 1 1 , 2 
1PID2 = ( I+U/2 
B I IP 102 ) = Y I I + 1) +Y ( 1) 

LL = I P 102+ 1 1 0B2 
BILL) = Y ( I>l)-Y{ I) 

90 9 CONTINUE 
RETURN 
END 
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T AU * 


ThETA 


?• 50 

• 25 

• 13 


• 50 
#25 

• 13 


• 50 

• 25 

• 13 


0*50 
3 * 2 5 

0 . L 3 


APPENDIX C 

ILLUSTRATIVE OUTPUT - POOLES 


3. 

0. 50 

1*00 

1.50 

2*00 

2.50 


5RSOMX 

- 




0# L673E+01 

0. 1673E+01 

0# 1 673E +0 i 

0 .1673E + 01 

0.1673E+01 

0. 1673E+01 

)• 1673E+01 

0. 1673E+QI 

0.1673S+01 

0 • 1 673 F + 0 1 

0* 16738+01 

0.167 3E +01 

0.L673E+01 

C-. 1673E+01 

0.1673E+01 

0.1673E+01 

0.1673E+O1 

0. 1673E+0 L 


AVEPSti 





0 • 8 256E + 00 

0.82S6E+00 

Q. 8 256E + 00 

0.8256E+00 

0.82565+00 

0.8256E+00 

0.8256E+ 3) 

C.8256E+00 

0.8256E+30 

0*8 25 6E+ 00 

0.8256E+00 

0.8256E+00 

0.B256E+03 

0.92565+00 

0* 8256E+00 

0 *8 256E+ 00 

Q.8256E+O0 

0.8256F+00 


VESGMX 





0. 3585 c: +01 

0.3 585E + 31 

0.3585E+01 

0 .3585E+01 

0* 35855 + 01 

0. 3585F+01 

3.3583E+01 

0.3585F+01 

0.3585E+01 

0.3585E+01 

0.3585c +01 

0.3585E+01 

0.3585E+01 

0.3585E+01 

0.3585E+01 

3 .3 5 85E + 0 1 

0.3585E+01 

0 . 3585E+01 


AyVESQ 





0.1 121E+01 

0. U21E + 01 

0. I12LE+0I 

0.1L2 1E+01 

•0.1121E+01 

0.112 IE +01 

0* 1 L2 IE +0 1 

0.U21E + 01 

0. 1121E + 01 

0.112 1 E + Q 1 

0.1 L21E + 01 

0.112 IE + 01 

0. L 12 IE +0 i 

0.1121E + 01 

0. 1121E+01 

0 • 1 12 1E + 0 1 

0.1121E+01 

0. 112 IE+C1 
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APPENDIX D 
PROGRAM MODEL 


f 

£ 1.- OECtARATIGNS AND INPUT 

OIRENSION REMARKT14). PSK0I. PHl<8>» TH£TAI6)» TAU( 61 »AIPHA1 11> » 
1VWCI256.6.6). 8 IASI 256 .6 .6 > . 6ILH< 16.6 *6 ) t Q£L<0) 

CC6M1N KK. VOBS(2561. 6(256) 

DAlA IALPHA<I).I*l.ll>/O«00l*0.OQj£»O.Q05*Q»0l»U.025»i)*05»0.l0» 
10.25.0. 50 .0.75. 1.0/ 

REACTS. 600 IREKARKI l I#L*1.14) 
wR I TE I 6. BO I ) (RERARKll 1*1*1.14) 

RFACI5.E24I IGNR. ME 
IT* 2**LGRh 
FIT* IT 

READ 15.025) KG. IPSI(K)*K=1»K6> 
mR I IE (6.842) IPS1 1 Kl. K- 1 .KG 1 
READ ( 5.625) KG. I PHUK) .K=l.KG) 

WRITE 16.0431 I PHI IK). K=1.KG1 
READ <5.0251 KG.iOEl <K>, K«1*KG) 

WRITE I E. 644 > I DEL ( K I . K=1.KG> 

*FALI5.625> LTh » ( THFTA C LI. 1*1. LTFi) 

RfACtS.625) RTAU. ITAUIRI. K*l,MTAU) 

I ps i ** KG 
FIF a IP 
1PP1 = IP ♦ I 
C 

f 2.- HOCEl AND FITTING 

C 

DO A 9 K* 1 «RTAU 

DO 48 l-I.LTH 

00 20 1=1. IP 

A= 1.0 

C = 0.0 

OG 15 K=1.KG 

KM1* K-l 

J= 2*+KMl 

AX = AND! < 1-1 > . J1 

XI = 2*IARSIKH1.AX)-1 

A* A4PSUKHXI 

C = C + PhHKtKX I-0£LlK) 1**2 
15 CONTINUE 

/mjlI.L.Rl = .THETAU.)*<A+TAU<*)*EXP<-C>) 

B IASI l.L .* * * 0.0 
2C CCNTINUE 
46 CONTINUE 
4 9 COAT IMjE 

40 READ 15.024) RP.KP.KF 

f 

f 5.- PREDICTIONS 

f 

00 690 pal.MTAU 
00 680 L=1.LTH 
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KK = LOME 





I F 1 KG 

«E£. 

4) 

GO 

TO 

204 

IF IKG 

• EC. 

bl 

GO 

TO 

205 

IF I KG 

♦ec. 

61 

GO 

TO 

. 2 CO 

IF IKG 

« EC • 

7) 

GO 

TO 

207 

IF 1 KG 

• EC. 


GO 

TO 

208 


204 nr 214 i*i « it 

YCBSI I ) = YMUI f .1 ,M) 
2 14 CONTINUE 
C At I VATFS 
on 344 1 = 1 , IT 
dll) * BID/FIT 
BIIMI.MI = HI I) 
349 CONTINUE 

KK * KG 
00 546 1=1, IP 

IPF1MI = IPP1-I 
YCBSI I) = 8IIPP1BI) 
546 r.CM I NUE 
CALI YAIFS 


GO TO 

tCC 



205 YOB SI 

l > 

si 

YMUI l.L.M) 

YOFSI 

; ) 

Si 

YMUt 2 • L • M ) 

YOB SI 

3) 

= 

YMUI 19*L*M) 

YOB SI 

4> 

• 

YMUI 20.L,M) 

YfiftSI 

5) 

sc 

YMU 1 2 1 *L« M ) 

YOBS 1 

6 1 

Si 

YMUI22 *L ♦ M) 

YOB SI 

11 

* 

YMUI 7 ,L , M) 

YOBS I 

*) 

- 

YMUI b ,L » M ) 

YOB SI 

91 


YMUI 25.L,M) 

VCBSI 10) 


YMUI 26,L«M) 

YOHSI 1 1 ) 

JS 

YMUI 11 .L«M> 

YOFSI 12) 

s 

YMUI 12.L.M) 

YC8SI 13) 

3 

YMUI 13 ,t,M) 

YOBSI 14) 

£ 

YMUI |4,L» M) 

YOBS 1 16 ) 


YMIJI 3 1 « L * M ) 

YOBSI 16) 

£ 

YMU(32,L,MI 


CALL VATFS 
00 359 1=1, IT 
HI I ) = Bl 1 I/FIT 
HHMI.LiM) = BID 
359 CONTI NOE 
KK = KG 

on 95 1 1=1,20 

55 1 YOBSI 1 1 = 0,0 
YOB S I 13) = Bl 14) 
YOFSI 14* = Bl 13) 
Y0BSI15* = HI 16) 

YOB S I 16) = HI 1 5) 

00 552 1=2 1,32 
IPPIMI * 33-1 

552 YOB S I I > * Bl IPPIMI) 
CALL YATES 
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GO TO 

6C0 


?C6 YOHS( 

1 ) 

JS 

YMU 1 1,L*8> 

YOBS< 

i > 

X 

YMU ( 34 *L .8) 

YOPS( 

3 ) 

= 

YBU( 19.L.M) 

Y08S( 

4) 

X 

YMU ( 52 *L » 8 > 

YOB S( 


X 

YMU (53*1*8) 

Y08S( 

6 > 

X 

V8U( 22, L ,8 ) 

YOBS( 

3> 

- 

YMUl 39 *L,M) 

YObSI 

B I 

X 

YMUl 8»L,8J 

YCflS( 

91 

X 

Y81J( 5 7 • L * 8 ) 

Y08S( 10) 


YMU ( 26 * L ,8 > 

YOfl S( 11 > 

X 

YMIJ( 43 »L.M> 

YOB S ( 17) 

X 

YMU ( J2 *L .8) 

YOB S ( 13> 

— 

YMU ( 13*L*M) 

YOBS < 14) 

x 

Y8U( 46 *L • 8 ) 

YOB S C 1 5 ) 

X 

YMU ( 31 *L • 8 ) 

vri8.s< it » 

X 

Y81) ( 64 »L ♦ 8) 


CALL YATES 
00 3 b* 1 = 1 * f T 
8(1) * 81 D/FIT 
dllMI,l.NI= B( I) 
369 CC6TIKUE 
KK = KG 
00 561 1*1*5? 

56 1 YCHS( I ) = 0-0 

YG8S131) * 8(13) 
YCPS l 32 ) * 6(1*) 

YOBS (A?) * 8(16) 
YfiPSI A 8 ) * 8(15) 
f)C 56? 1 = 5 3*6* 
(PF 181 * 65 -C 
56? Y08S( I ) * Bl 1PP18 I ) 
CALL YATES 
GO TO 600 


? 0 ? YOB S ( 

1 1 

X 

Y8U ( 

l*L*M) 

Yces( 

? ) 

X 

YMU( 

98*1*8) 

YCBSI 

3 > 

X 

YMU 1 

83, L *8) 

YG8S( 

4) 

X 

YMUC 

52,L«M) 

YOHS( 

5J 

X 

YMU( 

53, t ,MI 

Y C B S ( 

t > 

X 

YMUC 

66. L ,8) 

YOB S ( 

?> 

X 

YMU4 10 3* L*M ) 

YOB S( 

8) 

= 

vwui 

B,L .8) 

Y08SI 

9) 

X 

YMU( 12 1 * L *8) 

YOP S( 10) 

= 

YMU< 

26, L. 8) 

Y06S( ID 

X 

YMU( 

43, L, 8) 

YOBS( 12) 

X 

YMU 1 

76. L ,8) 

YOBS 4 13 ) 

X 

Y8U( 

77, L, 8) 

Y06S( 14) 

X 

YMU( 

46,1,8) 

Y06S( 15) 

s 

VMU( 

3 1 »l .8 > 

YOB S< It) 

X 

YMU( 128, L. 8) 


CALL YATES 
00 37* 1=1* IT 
B ( I > = bi I l/FIT 
BUM I ,L *8 ) = H( I) 
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4/S CCATiMiF 
KK a KG 
or 571 1-1.117 

5/1 VCBSl I * - 0.0 

YOPS<64> = H(UI 
YC8SI951 * 8(131 
YCP S ( 96 i - 8114) 

YOPS I 1111= 8(161 
YOB S I 1121s fit 151 
00 5/2 1— 1 18. 126 
IPPIMl * 12S-I 

si ? vokisi r i * b< ippimii 

CALI YATES 
GO TO 60C 
f 

206 YGBSI 1) = YMU< l.l.Ml 
YOBSi 2) * YMU(226.L.M> 

YC8S1 31 * YMUI 21 l.L .M 1 
YCBS( 41 * YMUI 52.L.M1 
YOB S( 51 * YMUU8WI.B1 
YOBSI 61 = YMU ( Sfc.l. Ml 
Y08S1 71 * YMU ( 103. L. Ml 
YOB SI 81 * YMU ( 1 3* * L * M 1 
YOBS( SI = YMUU21.L.M1 
YOB S < 101 * YMUI 1 54, L, Ml 
YOBS 1 11) * YMUU71.L.M1 
YCBSIWl * VMU( 76.L.M1 
YOBS ( 13 1 * YMU< 205. L.M > 

Y0BSI141 * YMU t 46. L .Ml 
Y0BSU5) * YMUl 31.L.M1 
Y08S( 16 1 * YMUl 256. L.M) 
f.AI L YATES 
00 38S 1*1. IT 
ft! I 1 '* HU 1/E I T 
BILK l.L. Ml* «< (1 

385 CONTINUE 
KK = KG 

00 581 1*1.245 

581 YOB S ( 1 1 * C.O 
YOBS! 1281 * BI81 
YOBS( IS2 1 * H< 121 
YOBS ( 223 > * B( 131 
YCPS( 2241 * 8(141 
YOBSI 239) * 6( 161 
YOBS( 2401 * B ( 151 
00 582 1*246,256 
IPPIMl * 257-1 

582 VCBS( II * PIIPP1MU 
YC8S( 2491 * 0.0 
CALL YATES 

GO TO 600 
C 

600 00 609 1*1* IP 

IPPIMl * IfPl-l 

BIASU.l.M! * BIIPP1MI 1-YMUI 1 , L .Ml 

60S CCNTINUF 
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6*iC r.GM inue 

6 SC CCNT INUE 
f 

f 8.- (lliTPtil 

C 

WRITE (to. 830) I7*NE*K6«MP.ALPhA(KP>*ALPHA.iKfl 
WHITE I to* 832 > (TAUM).M=1 .HTAU) 

WRITE 16*838) 

WRITE 46,839) 1 1 THETA! L) * I * 4 YMU4 i * L*M> ,H=i # HTAU> » I =i » I P > » L-ii LTH> 
WRITE 46,832! I TAG! M ) ,M« 1 ♦ MTAU > 

WRITE (£.84 0) 

WRITE 46, 839 > 4 4 THETA 4 1 1 , I * 4B2LM4 I »L »M) .M=i,MT AU > » I = 1 » IT) , L= i , LTH) 
wRITE 46, 832 > ! TAU4M) ,M=1,M1AU) 

WRITE 46*84! ) 

WR ITfc 46*8391 (4TFfcTA(4_),I«(8lAi»4 4,L»Ml, W = 1 , MT AU ) * I s l»IP)»L s l»ilH) 

800 FORMAT 4 ! 3A6, A2 > 

801 FORMAT 41M * 10X . 1 3A6 . A2/ / ) 

824 FORMAT 4315) 

825 FORMAT 4 15.865*0) 

83C FORMAT 4JH0,3X,4HlT *l5»SX*4Hf*E * I 5 »3X *4HKG *I5,6X»4HMP =I5»5X» 
18HALPHAP af6.3. 5K»8HALPhAF «E6*3//1 
8 32 FORMAT 41H ,4HTAU=6F 1 3 *2 / /) 

838 FORMAT 4 1H0 . 5HTHETA. IX . 1H I . 3X * 3H Y M 0/ i > 

839 FORMAT 4 1FO.F4.0* I4.6E14.4! 

84C FORMAT 4 1F0.5HTHETA. 1 X *IH I *3X * 4HB 4 LM/ / ! 

84! FORMAT 4 1 FC.5HTHE TA. IX . 1H I *3X* 4H6I AS // > 

842 FORMAT 4 1FC , 4HPS I=8E |4.4 ! 

843 FORMAT 4 1H0. 4HPH I «8E 14 .4 > 

844 FORMAT 4 1F0 .4FOE4. =8£ 14*4! 



SlieRnuTIM? YATES 
Cl'MHQN KK.Y(?S6),8<25t>) 
1 | a >**KK 
1 IC82 - 11/2 
KKK I = KK-1 

nn SC 8 K« 1 .KKM1 
i)0 906 1=1,11 ,2 
IPJC7 * ( 1*1 i/2 
HI tPini » * V( 1+1)+Y< I) 
LI = IP1C2+IICB2 
SO* hUl ) = Y< I* 1 ) — Y( 1) 

no sc? 1=1, u 

SO? Y ( n a 8(1) 

SC* CFMINUF 

no scs i*i. u ,2 

[PIC/ * ( 1 + 1 >/2 

HI I P 102 ) * Vi i+H+Yl 1) 

LI * IP102+ 1 108/ 

HUL ) * Y( I ♦ l V < I) 

SOS COM INL€ 

HE TURN 
END 
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TABLE I. - SAMPLE FRACTILES FOR ORDER STATISTICS 
FROM SAMPLES OF SIZE g 











TABLE II. - VALUES OF ^ 


k 

g 


4 

5 

6 

7 

8 

1 

1.00 

1.00 

1.00 

1.00 

1.00 

2 

.65 

.67 

.72 

.75 

.76 

3 

.41 

.45 

.54 

.58 

.63 

4 

.20 

.28 

.39 

.44 

.47 

5 


.14 

.26 

.32 

.37 

6 



.12 

.20 

.27 

7 




.10 

.17 

8 




i 

.07 







TABLE III. - POINTS OF INFLECTION (EQ. (24)) 


[*k " U 






TABLE IV. - GENERATORS OF EXPERIMENT PLANS 




TABLE V. - DEFINING CONTRASTS 


g 

5 

6 

7 

8 

Xq 

X 0 

X 0 

x o 

X 2 X 3 X 4 X 5 

X 2 X 3 X 4 X 5 

X 2 X 3 X 4 X 5 

X 2 X 3 X 4 X 5 j 

i 

X 1 X 3 X 4 X 6 

X 1 X 3 X 4 X 6 

X 1 X 3 X 4 X 6 


W5 X 6 

X L X 2 X 5 X 6 

X 1 X 2 X 5 X 6 



X 1 X 2 X 4 X 7 

x x x 2 x 4 x 7 



X 1 X 3 X 5 X 7 

X 1 X 3 X 5 X 7 



X 2 X 3 X 6 X 7 

X 2 X 3 X 6 X 7 1 



X 4 X 5 X 6 X 7 

X 4 X 5 X 6 X 7 

X 1 X 2 X 3 Xg 


! 

1 

X 1 X 4 X 5 X 8 
X 2 X 4 X 6 X 8 
X 3 X 5 X 6 X 8 
X 3 X 4 X 7 X 8 
I X 2 X 5 X 7 X 8 
1 X 1 X 6 X 7 X 8 
1 X 1 X 2 X 3 X 4 X 5 X 6 X 7 X 8 





TABLE VII. - YATES 


ORDER OF TREATMENTS 


1 

(1) 

33 f 

65 g 

97 fg 

129 h 

161 

fh 

193 gh 

225 f gh 

2 

a 

34 af 

66 ag 

98 afg 

130 ah 

162 

afh 

194 agh 

226 af gh 

3 

b 

35 bf 

67 bg 

99 bfg 

131 bh 

163 

bfh 

195 bgh 

227 bf gh 

4 

ab 

36 abf 

68 abg 

100 ab fg 

132 abh 

164 

abfh 

196 abgh 

228 ab f gh 

5 

c 

37 cf 

69 eg 

101 cfg 

133 eh 

165 

cfh 

197 cgh 

229 cfgh 

6 

ac 

38 acf 

70 aeg 

102 acfg 

134 ach 

166 

acfh 

198 aegh 

230 acfgh 

7 

be 

39 bef 

71 beg 

103 befg 

135 bch 

167 

befh 

199 begh 

231 bcfgh , 

8 

abc 

40 abef 

72 abeg 

104 abefg 

136 abch 

168 

abefh 

■200 abegh 

232 abef gh 

9 

d 

41 df 

73 dg 

105 dfg 

137 dh 

169 

dfh 

201 dgh 

233 dfgh 

10 

ad 

42 adf 

74 adg 

106 adfg 

138 adh 

170 

adfh 

202 adgh 

234 adfgh 

11 

bd 

43 bdf 

75 bdg 

107 bdf g 

139 bdh 

171 

bdfh 

203 bdgh 

235 bdfgh 

12 

abd 

44 abdf 

76 abdg 

108 abdfg 

140 abdh 

172 

abdfh 

204 abdg 

236 abdfgh 

13 

cd 

45 cdf 

77 edg 

109 cdfg 

141 cdh 

173 

cdfh 

205 edgh 

•237 cdf gh 

14 

acd 

46 aedf 

78 aedg 

110 aedfg 

142 aedh 

174 

acd f h 

206 aedgh 

238 acdfgh 

15 

bed 

47 bedf 

79 bedg 

111 bedfg 

143 bedh 

175 

bedfh 

207 bedgh 

239 bedf gh 

16 

abed 

48 abedf 

80 abedg 

112 abedfg 

144 abedh 

176 

abedfh 

208 abedgh 

240 abcdfgh 

17 

e 

49 ef 

81 eg 

113 efg 

145 eh 

177 

efh 

209 egh 

241 e f gh 

18 

ae 

50 aef 

82 aeg 

114 aefg 

146 aeh 

178 

ae fh 

210 aegh 

242 aefgh 

19 

be 

51 bef 

83 beg 

115 befg 

147 beh 

179 

befh 

211 begh 

243 bef gh 

20 

abe 

52 abef 

84 abeg 

116 abefg 

148 abeh 

180 abefh 

212 abegh 

244 abefgh 

21 

ce 

53 cef 

85 ceg 

117 cef g 

149 ceh 

181 

cefh 

213 cegh 

245 cefgh 

22 

ace 

54 acef 

86 aceg 

118 acefg 

150 aceh 

182 

acef h 

214 acegh 

246 a cefgh 

23 

bee 

55 beef 

87 bceg 

119 bcefg 

151 bceh 

183 

bcefh 

215 bcegh 

247 bcefgh 

24 

abce 

56 abcef 

88 abceg 

120 abcefg 

152 abceh 

184 

abcef h 

216 abcegh 

248 abcef gh 

25 

de 

57 def 

89 deg 

121 defg 

153 deh 

185 

defh 

217 degh 

249 def gh 

26 

ade 

58 adef 

90 adeg 

122 ade fg 

154 a deh 

186 

adefh 

218 adegh 

250 adefgh 

27 

bde 

59 bdef 

91 bdeg 

123 bdef g 

155 bdeh 

187 

bde fh 

219 bdegh 

251 bdefgh 

28 

abde 

60 abdef 

92 abdeg 

124 abdefg 

156 abdeh 

188 

abdefh 

220 abdegh 

252 abdefgh 

29 

ede 

61 edef 

93 edeg 

125 ede fg 

157 edeh 

189 

edefh 

221 edegh 

253 ede f gh 

30 

aede 

62 aedef 

94 aedeg 

126 aedefg 

158 a edeh 

190 

aede fh 

222 aedegh 

254 acdefgh 

31 

bede 

63 bedef 

95 bedeg 

127 b edef g 

159 bedeh 

191 

bede fh 

223 b edegh 

255 bedef gh 

32 

abede 

64 abede f 

96 abede g 

128 abede fg 

160 abedeh 

192 

abede fh 

224 ab edegh 

256 abede fgh 



TABLE vrir. - TREATMENTS OF EXPERIMENT PLANS AND ALIASED COMBINATIONS OF LOWER ORDER PARAMETERS 


g 4 5 

h 0 I 


6 

2 


7 

3 


8 

4 


1 

(1) 

0q 

1 

(1) 

60 

1 

U) 

0Q 

1 

U) 

*0 


1 

(1) 

0o 

2 

a 

*i 

2 

a 

*1 

34 

af 

6l 

98 

afg 

S 1 


226 

afgh 

01 

3 

b 

H 

19 

be 

h 

19 

be 

0 2 

83 

beg 

0 2 


211 

begh 

02 

4 

ab 

0 12 

20 

abe 

012 

52 

abef 

6l2 + 656 

52 

abef 

612+647 + 656 


52 

abef 

012+ 047 + 038 +056 

5 

c 

0 3 

21 

ce 

63 

53 

cef 

63 

53 

cef 

83 


181 

cefh 

63 

6 

ac 

&13 

22 

ace 

e 13 

22 

ace 

013 + 046 

86 

aceg 

013+ 646 + 057 


86 

aceg 

013 + 046+ e 57+ 028 

7 

be 

e 23 

7 

be 

623 + 645 

39 

bef 

623 + 645 

103 

befg 

0 23 + 0 45 + 0 67 


103 

befg 

023 + 045+ 067+ 0 18 

a 

abc 

0123 

8 

abe 

6123+6145 

8 

abc 

6123 + 6145 +6356 + 6246 

8 

abc 

0123 + 6145 + 0246 
+ 0356 +0257 

+ b 347 
+ b 167 

136 

abch 

0123 + 08 

9 

d 

04 

25 

de 

64 

57 

def 

64 

121 

defg 

0 4 


121 

defg 

04 

10 

ad 

014 

26 

ade 

014 

26 

ade 

014+ 636 

26 

ade 

014+ 036+ 027 


154 

adeh 

014+ 036+ 027+ 058 

u 

bd 

024 

11 

bd 

624 + 635 

43 

bdf 

624+ 635 

43 

bdf 

024 + 035+ 017 


171 

bdfb 

024 + 635+ 017+ 068 

12 

abd 

^124 

12 

abd 

6124 + 6135 

12 

abd 

6124+ 6135 + 6 2 36 + 0456 

76 

abdg 

0124+ 67 


76 

abdg 

0124+ 07 

13 

cd 

634 

13 

cd 

634 + s 25 

13 

cd 

034+ 025 +616 

77 

edg 

034+ 025+616 


205 

edgh 

034+ 025+ 016+ 678 

14 

acd 

e 134 

14 

acd 

0134+6125 

46 

aedf 

0X34 + 06 

46 

aedf 

0134 + 06 


46 

aedf 

0134 + 06 

15 

bed 

0234 

31 

bede 

6234 +65 

31 

bede 

6234 + 65 

31 

bede 

0234 +05 


31 

bede 

0234+ 05 

16 

abed 

61234 

32 

abede 

61234 +615 

64 

abedef 

61234 + 615 +026 

128 

abede fg 

01234 + 015 + 026 "• 

■ 637 

256 

abede fgh 

01234 + 015 + 026 + 037 +048 


Numbers In columns are the Yates order number of the treatment in a full factorial experiment, as in table VIL 



TABLE IX, - STARTING MODELS FOR SUBSET REGRESSION PROCEDURES 


[The number i is the order clumber of the Yates estimate for a full factorial experiment,} 

g => 4, h » 0 

i 

12 3 4 5 6 7 S 9 10 11 12 13 14 15 16 

y * +$2 x 2 + 0 12 x 1 x 2 * 0 3 x 3 + ^13 x 1 x 3 + ^23 x 2*3 +B 123 X 1 X 2 x 3 + ^4 x 4 +0 14 x 1 x 4 +& 24 x 2 x 4 + 0 124 X Z X 2 X 4 + 0 34 x 3 x 4 +6 134 x 1 x 3 x 4 +0 234 X 2 X 3 X 4 +0 1234 X 1 X 2 X 3 X 4 

g » 5, h » l 


l 2 3 4 5 6 7 8 9 10 11 12 19 20 17 18 

y ^ 3 q + ®l x l + ^2 x 2 +0 12 x l x 2 + ^3 x 3 + ^13 X 1 X 3 + ^23 X 2 X 3 +0 123 X 1 X 2 X 3 +0 4 x 4 +0 14 x l x 4 +0 24 X 2 X 4 + ^124 *i x 2 x 4 +0 25 X 2 X 5 +0 125 X 1 X 2 X 5 + ^5 X 5 +0 15 x l x 5 

g - 6, h = 2 
i 


17 


1 2 3 4 5 6 


129 


34 33 


17 


13 


1 2 3 4 5 6 7 3 9 10 11 12 34 33 

y ^ 3 Q +S i x i + ^2 X 2 +6 12 x l x 2 + ^3 x 3 +e i3 x l x 3 + ^23 X 2 X 3 +0 123 X 1 X 2 X 3 ^4 X 4 +6 14 X 1 X 4 +6 24 X 2 X 4 +0 124 X 1 X 2 X 4 +6 16 X 1 X 6 +0 6 x 6 +*5 X 5 +0 15 x l x 5 

g = 7, h - 3 
i 

1 2 3 4 5 6 7 8 9 10 11 65 34 33 17 18 

y - 6 q +B j x l +B 2 x 2 +a 12 x 1 x 2 ^3X3 +B 13 x 1 x 3 +0 12 J x 1 x 2 x 3 +6^ +8^*^ +B 24 x £ x 4 +8^ +*16*1*6 +3 6 x 6 +(3 5 x 5 +3 15 2 l x 5 

g = 8, h » 4 
i 

9 10 11 65 


18 


y 86 & 0 +0 x x L +0 2 x 2 +0 12 X 1 X 2 +0 3 x 3 +0 13 X 1 X 3 "**23*2*3 +0 8 X 8 "**4*4 +6 14 X 1 X 4 +e 24 X 2 X 4 * 0 7 X 7 +0 16 x l x 6 +0 6 x 6 +S 5 X 5 +0 15 x l x 5 



TABLE X. - COMPUTER PROGRAMS 


Name 

a 2 

a p a £ 

n e 

Output 

MODEL 

0 

0 1.0 1.0 


YMU(I,L,M) I - 1,IP 
BILM(I,L,M) I = 1,IT 
BIAS(I,L,M) I = 1,1P 

POOLES 

0 

Op tional 


ERSQMX (L,M) 
AVERSQ (L,M) 

| POOLES 

1 

Optional 

Optional 

ERSQMX (L,M) 
AVERSQ (L,M) 
VESQMX <L,M) 
AWESQ (L,M) 










TABLE XII. - SUMMARY OF POPULATION CHARACTERISTICS AS 

determined with program model 

[Values of i are Yates order number of treatment 
at which maximum absolute values of mean and 
bias occurred.] 



























TABLE XIII. - OPTIMAL STRATEGIES FOR GIVEN 0 AND t 


(a) x = 0.0; 6 k = -1, 1, -1, -1, . - • 5 = 1000 


e 

S 

4 

5 

6 

7 

8 


h 

0 

1 

2 

3 

4 

0.125 

Ct- 

1.00 

1.00 

1.00 

1.00 

1.00 


P 

"f 

.001 

.001 

.001 

.001 

.001 


e max 

.1444 

.1656 

.2086 

.2450 

.2843 



.0302 

.0357 

.0467 

.0560 

.0660 

.25 

Ctn 

1.00 

1.00 

.002 

.25 

.50 

P 

.001 

,001 

.002 

.001 

.005 


e 2 

.3855 

.4700 

.6411 

.7564 

.8794 


max 






V^max 

.0918 

.1133 

.1577 

.3418 

.9264 

.5 

°P 

a f 

.75 

.75 

.75 

1.00 

a 1.0 G 


.05 

.05 

,50 

.25 

1.00 


a 2 

^max 

.9423 

1.019 

1.092 

1.091 

1.117 



1.616 

1.645 

2.545 

2.551 

2.584 

1.0 

ct„ 

.75 

.75 

1.00 

1.00 

1.00 


P 

.10 

.10 

.25 

.25 

.50 


i 2 

.9798 

1.033 

1,100 

1.114 

1.113 


max 





V ^Ux 

1.973 

2.054 

2.436 

2.461 

2.612 

2.0 

a n 

.75 

.75 

1.00 

1.00 

.75 


P 

“f 

.io 

.10 

.25 

.25 

.75 



.9559 

.9912 

1.057 

1.108 

1.113 


) max 

1.859 

2.128 

2.450 

2.482 

| 

2.536 

4.0 

a i3 

.10 1 

.75 

.75 

.50 I 

1.00 


P 

.10 

.10 

.10 

.10 1 

.25 


e 2 

“max 

[ ,7964 

.9573 

.9769 

1.056 

1.069 


V<^W 

,1.407 

2.006 

2.094 

2.446 

2.542 

8,0 

a n 

' .025 

.10 

.10 

.50 

.25 


H 

“f 

! .025 

.10 

.10 

.05 

.25 


e 2 

pax 

.4786 

.7731 

.8800 

.9458 

1.016 


V <* 2 W 

.7771 

1.458 

1.655 

2.060 

2.444 

16.0 


.005 

.01 

.025 

.05 

.50 


a f 

.005 

.01 

.025 

.05 

.05 



.3447 

.4337 

.5642 

.7135 

.9346 


VC^W 

.2322 

.4721 

.7246 

1.356 

2.002 

32.0 

°D 

,005 

.01 

.001 

.005 

.025 


P 

“f 

,005 

.002 

.001 

.002 

.025 


e 2 

e tnax 

.3447 

.4177 

.5038 

.5745 

.6793 


V(? 2 )max 

.2322 

.3717 

.4920 

.6628 

.9404 



TABLE XIII. - Continued. OPTIMAL STRATEGIES FOR GIVEN 6 AND T 
(b) T = 1.0, 6 k “ -1, 1, -1. -1. - • -I "e ” 1000 


6 

g 

h 

4 

0 

5 ! 

1 

6 

2 

7 

3 

8 

4 

0.125 

«p | 

«f i 

1.00 

.001 

1.00 

.001 

1.00 

.001 

1.00 

.001 

1.00 

.001 


e max 

.1441 

.1624 

.2049 

.2427 

.2828 


V(e 2 W 

.0310 

.0358 

.0463 

.0552 

.0649 

.25 

°P 

Of 

.002 

.002 

.002 

.002 

.002 

.001 

.25 

.001 

.25 

.005 



.3883 

.4623 

.6257 

.7494 

.8850 


V(e 2 ) nax 

.1184 

.1139 

.1566 

.4531 

.8731 

.5 

“P 

“f 

.75 

.05 

.50 

.05 

.75 

.25 

1.00 

.25 

1.00 

.25 


t 1 

e max 

.9296 

,9879 

i .078 

1.088 

1.149 



1.617 

1.671 

2.511 

2.544 

2.499 

1.0 

°p 

a f 

.75 

.10 

.75 

.10 

.75 

.10 

.75 

.10 

.75 

.10 


-z'l 

e max 

1.003 

1.076 

1.218 

1.314 

1.547 


V(c^)tnax 

2.224 

2 .196 

2.892 

3.984 

7.046 

2.0 

a„ 

1.00 

.25 

.25 

.25 

.50 


P 

a f 

.25 

.25 

.10 

.25 

.05 


e max 1 

1.068 

1.501 

2.183 

2 . 810 

3.106 



2.268 

3.832 

17.83 

22.51 

19.91 

4.0 

a 

.75 

.10 

.10 

.25 

.25 


4 

: * 75 

.05 

.01 

.01 

.01 



1.112 

4.118 

6.206 

8.287 

9.033 


max 




379.3 



j 2.267 

23.59 

189.6 

160.0 

8.0 

<*_ 

j a 1.00 

.10 

.025 

.05 

.10 

p 

a f 

1,00 

.002 

.002 

.002 

.005 


e max 

! 1.116 

15.08 

24.99 

29.08 

39.62 


v ^ e2 ^max 

: 2.282 

198.1 

491.4x10 

232.2x10 

342.4 

16.0 

a „ 

a 1.00 

.025 

.005 

1.00 

.025 

P 

a f 

! 1.00 

.002 

.001 

.025 

.001 


e 2 

■max 

^ ) raax 

1.116 
. | 2.282 

56.11 
257 .9*10 

108.0 

160.4*lO 3 

159.8 

934.1x10 

177.9 

305.4*10 3 

32.0 

a« 

i 

1 a 1.00 

1.00 

a 1.00 

1.00 

.005 

P 

a £ 

1.00 

.75 

1.00 

.025 

.001 


i 2 

^maa 

V^maa 

. 1.116 

236.4 

462.8 

681.0 

842.0 


t | 2.282 

974.1 

196.4x10 

439.2xl0 : 

■ 258.8x10* 


»0. In all other cases, flip - 1. 



TABLE XIII. - Continued. OPTIMAL STRATEGIES FOR GIVEN G AND t 



(c) 1 - 

2.0, 6 fe = 

-i, i. -i. 

-l, . . . ; r 

c = 1000 


6 


4 

5 

6 

7 

8 



0 

1 

2 

3 

4 

0.125 


1.00 

1.00 

1,00 

1.00 

1.00 


a f 

.001 

.001 

.001 

.001 

.001 


e max 

.1473 

.1629 

.2012 

.2403 

.2814 


v ( e2 )max 

.0319 

,0360 

.0459 

.0545 

.0639 

.25 

ct n 

.001 

.005 

.005 

.25 

.25 


P 

u f 

.001 

.005 

.005 

.001 

.002 


p 

max 

.4037 

.4643 

.6146 

.7412 

.8586 


v ( e2 )jnax 

.0984 

.1571 

. 1646 

.4785 

.6676 

.5 

CL 

. .75 

1.00 

1.00 

.50 

.50 


Of 

.05 

.10 1 

.10 

.05 

.10 



.9009 

.9760 

1.150 

1.257 

1.412 


X 

1 

CN 

Up 

> 

1.978 

1.998 

3.018 

2.859 

3.550 

1.0 

ou 

.75 

.75 

.75 

,25 

.75 


p 

a f 

.10 

.05 

.05 

.25 

.05 


p 

e max 

1.015 

1.556 

2.035 

2.542 

2.726 


v ^ e ^^inax 

2.026 

3.788 

6.866 

12.25 

17.94 

2,0 

cl 

1.00 

,10 

.25 

1.00 

.25 


V 

*f 

1.00 

.10 

.025 

,05 

.025 


e max 

1.114 

3.618 

6.315 

7,691 

8.787 


^ (e 2 )ujax 

2.294 

26.72 

51.36 

143.6 

210.8 

4.0 

cl 

a 1.00 

.05 

.10 

.10 

.10 


P 

“f 

1.00 

.025 

.005 

.005 

.01 


e 2 

e max 

1.116 

12.27 

22.64 

26.16 

32.25 


V ^^max 

2.282 

52,58 

808.4 

135 . 5*10 

371.1*10 

8.0 

CL, 

a 1.00 

.10 

.025 

.05 , 

.05 


n? 

a f 

1.00 

.005 

,002 

.001 

.002 


e max 

1.116 

47.84 

100.4 

105,9 

144.8 


v(^W 

2.282 

801,8 

369.5*10 2 

292.9*10 2 

503.2xl0 2 

16.0 


a 1.00 

.10 

.025 

.025 

.025 


p 

Of 

1.00 

.025 

.001 

,001 

.001 


e 2 

max 

1.116 

192.9 

391.2 

488,5 

714.0 


v ( e2 )max 

2.282 

984.2x10 

117 ,8xl0 6 

473.3*L0 3 

587.3xl0 3 

32.0 

CL 

a 1.00 

.05 

.005 

1.00 

.005 


P 

1.00 

.05 

.001 

.025 

.002 



1.116 

819,5 

159.9x10 

228.0x10 

272.5x10 


V^nax 

2,282 

L12.4xl0 3 

587.7x10* 

101.8xl0 4 

308.0xl0 5 


“0. In all other cases = 1. 



TABLE XIU. - Continued . OPTIMAL STRATEGIES FOR GIVEN 9 AND T 


( d ) t = 0.0; <5 k = 1, 1, 1, 1, . . .; n , = 1000 




4 

5 

6 

7 

8 



0 

1 

2 

3 

4 

0-125 

*P 

n f 

1.00 

1.00 

1.00 

1.00 

1.00 


.001 

.001 

.001 

.001 

.001 


e * 

max 

.1444 

.1656 

.2086 

.2450 

.2843 


V ^W 

.0302 

.0357 

.0467 

.0560 

.0660 

.25 

a n 

1,00 

1.00 

.002 

.25 

.25 


P 

a f 

.001 

.001 

.002 

.001 

.002 


e 2 

.3855 

.4700 

.6411 

.7564 

.8930 


V <« 2 W 

.0918 

.1133 

.1577 

.3418 

.6352 

-5 

a _ 

.75 

.75 

.75 

1.00 

a 1.00 


P 

H 

.05 

.05 

.50 

.25 

1.00 


e 2 

^TTiax 

.9423 

1.019 

1.092 

1.091 

1.117 


VCe 2 )^ 

1.616 

1.645 

2.545 

2.5 51 

2.584 

1.0 

On 

.75 

.75 

1.00 

1.00 

1.00 


«S 

.10 

.10 

.25 

.25 

.50 


e 2 

.9798 

1.033 

1.100 

1.114 

1.113 


max 







V( e ^)[nax 

1.973 

2.054 

2.436 

2.461 

2.612 

2.0 

a t > 

. 75 ! 

.75 

1.00 

1.00 

.75 


P 

a f 

.10 

.10 

.25 

.25 

.75 


x 2 

e max 

.9559 

.9912 

1.057 

1.108 

1.113 


V ( e^) max 

1.859 

2.128 

2.450 

2.482 

2.536 

4.0 - 

a P 

.10 

.75 

.75 

.50 | 

1.00 


r 

a f 

.10 

.10 

.10 

.10 

.25 


i -2 

max 

. 7964 

.9573 

.9769 

1.056 

1.069 


V(^) m ax 

1.407 

2.006 

2.094 

2.446 

2.542 

8.0 

o D 

.025 

.10 

.10 

.50 

.25 



.025 

.10 

.10 

.05 

.25 


®max 

.4786 

.7731 

.8800 

.9458 

1.016 



.7771 

1.458 

1.655 

2.060 

2.444 

16.0 

% 

.005 

.01 

.025 

.05 

.50 


r 

*f 

.005 

.01 

.025 

.05 

.05 


e 2 

max 

.3447 

.4337 

.5642 

.7135 

.9346 


VCe 2 )^ 

.2322 

,4721 

.7246 

1.356 

2.002 

32.0 


.005 

.01 

.001 

.005 

.025 


r 

*f 

.005 

.002 

.001 

.002 

.025 


Tiax 

.3447 

.4177 

.5038 

.5745 

.6793 


v ^ e ^max 

.2322 

.3717 

.4920 

.6628 

.9404 


S Dip =* 0. In all other cases nip = 1. 




TABLE XIII* - Continued. 


OPTIMAL STRATEGIES FOR GIVEN 0 AND i 


<e) T - 1*0; \ 


. ; n * 100D 



l mp « 0. In all other cases ^ = 1 . 




TABLE XIII. - Concluded. OPTIMAL STRATEGIES FOR GIVEN 0 AND 


(f) T = 2.0; 6^ » 1, 1, 1, 1, 1, . . . ; n e =* 1000 


e 


4 

5 

6 

7 

8 



0 

1 

2 

3 

4 

0.125 


0.002 

0.001 

1.00 

1.00 

1.00 


<? 

.001 

.001 

.001 

.001 

.001 



.3188 

.3631 

.4398 

.4988 

.5592 



.0705 

.0814 

. 1002 

.1148 

.1298 

.25 

a P 

i50 j 

1.00 

.75 

.75 

1.00 


Of 

,025 

,10 

.10 

.10 

.25 


4a* 

.9223 

.9629 

1,018 

1.030 

1.089 


^( e ^max 

1.381 

1.840 

2.109 

2.150 

2.448 

.5 

a p 

1.00 1 

1.00 

1.00 

.75 

1.00 


4 

.25 ! 

.25 

.25 

.25 

.25 


e 2 

e max 

1.048 , 

1.132 

1.191 

1.220 

1.216 


^max 

2.080 

2.382 

2.594 

2.823 

2.958 

1.0 

a p 

1.00 

1.00 

1.00 

1.00 

1.00 


a £ 

.25 

.25 

.25 

.25 

.25 


C ^.ciX 

1.059 

1.414 

1,669 

1.716 

1.842 


v(i2 W 

2.215 

3.851 

5.150 

5.152 

5.435 



TABLE XIV. - VALUES OF g AND 6 AT WHICH STATED VALUES 


OF a AND C*_ WERE OPTIMAL 
P f 

(a) T « 0.0. 6 k - -I. 1, -1> -1, . * also - 1, 1, 1, 1, . . n e - 1000 



c See table XV(c) . 
d See table XV(d) . 










TABLE XIV 


Concluded, VALUES OF g AND 6 AT WHICH STATED 


VALUES OF a AND a. WERE OPTIMAL 
P f 

<b) t = 1,0; ^ ■ 1, 1, 1, 1, . . n ■ 1000 



g See table XV (g) , 
h See table XV(h), 




TABLE XV. - EFFECT OF a AND a, ON MEAN SQUARED ERROR AND ON VARIANCE OF MEAN SQUARED ERROR 
P f 


U) g - 4; 

h = 0; 

0 - 0.125; x = 

0.0; 6 k 

- -1, 1 

> -1, -1 

, and 

i ^ 

1, i, i; 

n = 1000 
e 

a 

P 


“f 



L.OO 

0.75 

0.50 

0.25 

0.10 

0.05 

0.025 

0.01 

0 . 005 

0.002 

0.001 

a 1.00 

e* 

max 

1.116 













2.282 











1.00 

e 2 

max 

1.116 

1.112 

1.103 

1.043 

0.8372 

0.6010 

0.4108 

0.2738 

0.2190 

0.1922 

10.1444] 



2.275 

2.294 

2.279 

2.091 

1.599 

1.155 

.6710 

.4263 

.2155 

.1681 

.0302 

.75 

iLx 


1.091 

1.085 

1.050 

[oTu^ 

.7818 

.5751 

.3835 

.3059 

.2525 

.1902 


v <e 2 >roax 


2.246 

2.251 

2.173 

1.881 

1.530 

1.105 

. 6344 

.4867 

.2804 

.1860 

.50 




1,020 

.9840 

.9194 

.8250 

.6943 

.5157 

.4000 

.3195 

.2487 





2.238 

2.162 

1.932 

1.653 

1.323 

.9057 

.6548 

.4335 

.3008 

.25 

e 2 

e max 




.7182 

.6618 

.6216 

.5810 

.4693 

.3887 

.3360 

.2646 






1.544 

1.467 

1.364 

1.146 

,9791 

.7019 

.5091 

.3498 

.10 

e2 

e max 





.4100 

.3867 

.3750 

.3429 

.3169 

.2794 

.2284 


VCe 2 )^ 





.8122 

.7263 

.6804 

.5710 

.5168 

.4220 

.2612 

.05 

e 2 

e max 






.2906 

.2815 

.2602 

.2472 

.2385 

.2077 








.4488 

.4109 

.3647 

.3364 

.2905 

.2180 

.025 

e 2 

e max 







.2230 

.2135 

.2069 

.2037 

.1772 


^( c ^max 







,2712 

.2530 

.2364 

.2547 

.1975 

.01 

e 2 

rruix 








.1855 

.1827 ; 

.1823 

.1629 


^^max 








.1613 

.1413 

.1448 

.1267 

,005 

e 2 

e max 









.1637 

.1634 

.1555 


VCe 2 )^ 









.1135 

.1057 

.1209 

.002 

?2 

max 


| 








.1561 

.1504 


v <e 2 >max 










.1038 

.1143 

.001 

F Lx 











.1465 


v ^ 2 'nax 








.. . — 1 



.0325 


a nip = 0. In all other cases rap - 1. 



TABLE XV. - Continued * EFFECT OF a p AND a f ON MEAN SQUARED 
ERROR AND ON VARIANCE OF MEAN SQUARED ERROR 


(b) 6 k 

- -1. 1, 

-1> “I 

* • 1 

\ x > 

1, . * 

* 

; h = 0i 

n * 1000; 6 -= 
e * 

1.0; T - 

0.0 

a 


a f 





1.00 

0.75 

0.50 

0.25 

0.10 

0.05 

0.025 

0.01 

0.005 

0.002 

0.001 

a 1.00 

e 2 

max 

1.116 

2.282 












4ax 

V ( e 2 ) ma x 

1.115 

2.282 

1.116 

2.305 

1.109 

2.290 

1.042 

2.138 

1.090 

2.345 

1.719 

4.149 

2.829 

5.757 

3.987 
5 .068 

4.557 

3.617 

4.792 

2.812 

5.184 

1.306 

.75 

e 2 

e max 

V(e^) I nax 


1.101 

1.088 

1.044 

1 .97981 

1.181 

1.804 

2.846 

3.512 

4.018 

4.555 


2.244 

2.263 

2.214 

1.973 

2,574 

4.404 

5.831 

5.659 

4.860 

3.597 

.50 

x2 

e max 

V( e ^)[nax 



1.034 

2.191 

1.053 

2.167 

1.100 

2,169 

1.150 

2.447 

1.410 

3.446 

2.129 

5.075 

2.684 

5,795 

3.224 

1,254 

3.815 

5.321 

.25 

e 2 

max 




1.204 

2.470 

1.430 

3.247 

1.643 

3.868 

1.858 

4,375 

2.213 

5.138 

2,619 

5.704 

3.007 

1.212 

3.532 

5.648 

.10 

e 2 

e max 





2.194 

4.994 

2.585 

5.577 

2.896 

5.664 

3.228 

5,822 

3.524 

5.718 

3.768 

5.459 

4.168 

4.691 

.05 

F 2 

e max 

»(e 2 ) u , 






3.072 

5,879 

3.402 

5.618 

3.766 

5.311 

3.993 

5.002 

4.219 

4.551 

4.523 

3.704 

.025 

4a* 

V<* 2 W 







3.834 

5.203 

4.203 

4.458 

4,425 

4.003 

4.617 

3.504 

4.813 

2,818 

.01 

e 2 

max 

V (e^max 








4.448 

3.863 

4,670 

3.289 

4.839 

2,748 

4.968 

2.248 

.005 

e 2 

e max 

V (e 2 ) max 









4,769 

2.943 

4.929 ! 
2,382 

5.025 

2.003 

.002 

e 2 

c max 

v (e 2 )max 










4.968 

2.220 

5.065 

1.831 

.001 

e 2 

max 

V(e 2 ) raax 











5.086 

1.747 


□ip = 0. In all other cases, nip - 


1 




TABLE XV. 


- Continued. . EFFECT OF a p AND fl f ON MEAN SQUARED ERROR AND ON VARIANCE OF MEAN SQUARED ERROR 
(c) g - 4; h = 0;"e - 32.0; i - 0,0; S k - -1, 1, -1, -1; and = 1, 1» 1» 1; n e = 1000 


a 

a f 


P 

1.00 

0.75 

0.50 

0.25 

O.10 

0,05 

0.025 

0 0.01 

0.005 

0.002 

0.001 

a 1.00 F 2 ^ 

V(e 2 )max 

1.00 e aax 

v (e 2 )max 

• ^ e raax 

VCe 2 )^ 

^50 e 2 ^ 

. 25 ^Lax 
.10 ' 

V(e 2 ) ma x 
*05 e£ax 

v<e 2 W 

•025 T 2 ^ 

1 

•01 e max 

.005 e^a X 

vC^W 

.002 e max 

VCe^Jjpax 

>001 ^nax 

^max 

1.116 

2.282 

1.115 

2.277 

1.114 

2.297 

1.096 

2.250 

1.099 

2.284 

1.080 

2.247 

1.025 

2.200 

1.024 

2.127 

1.031 

2.233 

1.001 

2.200 

.8417 

1.664 

0.8100 

1.463 

1.893 

.9076 

1.921 

.7824 

1.560 

.5349 

1.066 

0,6388 

1.126 

.7814 

1,510 

.8203 

1.606 

.7362 

1,386 

.5632 

.8626 

.4589 

.5630 

I 

0.6226 

6.907 

.6200 

1.040 

.7104 

1.201 

,6677 

1.152 

.5361 

.7692 

.4448 

.5233 

.4032 

.4001 

8.046 

1.478*10 3 

1.413 

4.169*10 

.6322 

5.160 

.5928 

.8564 

.5130 

.6844 

.4312 

.4554 

.3974 

,3854 

.3643 

.2951 

1.222xl0 2 

1.967*10 5 

2.238x10 

1.430x10* 

3.494 

5.497*10 2 

.5934 

5.120 

.4608 

.5691 

.4086 

.4123 

.3815 

.3591 

.3552 

.2784 

|.3447 | 
.2322 

5,293xl0 2 

1,468*10 6 

1.232X10 2 

2.252xl0 5 

1.907x10 

1,367*10* 

1.140 

3.297*10 

.4174 

,4403 

.3901 

.3769 

.3739 ■ 

.3335 

.3535 

.2780 

.3450 

.2329 

.3458 

.2321 

5.231*10 3 

1.397xl0 3 

3.594xl0 3 
5 . 890 xlO 6 

2.265*10 3 

6.723*10 6 

1. 125 x 10 3 
4.622*10 6 

4.346*10 2 

2.085*10 6 

2.514*10 2 

1.252xl0 6 

1.102xl0 2 

5.630X10 5 

3,698*10 

1.909*10 5 

1,078*10 

5,484*10* 

1,078*10 
5.484 xlO* 

,3453 

.2024 


** 0. In all other cases trip - 1 





^ □ 0. In all Other cases rap *“ 1- 



TABLE XV. - Continued. EFFECT OF AND ON MEAN SQUARED 

ERROR AND ON VARIANCE OF MEAN SQUARED ERROR 




e) g - 

4, h - 

o, e = 

0,125; 

t - 1.0; 

*k- *• 

1, 1, 1 

; n = 1000 
' e 



a 

P 

■« 

1.00 

0.75 

0.50 

0.25 

0.10 

0.05 

0,025 

0.01 

0.005 

0.002 

0,001 

H.oo 

*2 

e max 

1.116 












V(®^)max 

2.282 











1,00 

e 2 

max 

1.115 

1.112 

1.101 

1.045 

0.8551 

0.6207 

0.4286 

0.3335 

0.2520 

0.2466 

[o . 2195 | 


VC^max 

2.291 

2.311 

2.289 

2,120 

1.653 

1.168 

.8042 

.4406 

.2171 

.1405 

.0460 

.75 

F 2 

e max 


1.089 

1.084 

1.056 


.7833 

.5991 

.4371 

.3267 

.2927 

.2520 


v ( e2 )max 


2.275 

2.275 

2.201 

1.807 

1.406 

1.125 

.6759 

.4127 

.2603 

.1137 

.50 

e2 



1.029 

.9936 

.9239 

,8327 

.6968 

.5251 

.3920 

.3454 

.2889 


V (« 2 >max 



2.214 

2.142 

1.842 

1.690 

1.358 

,9613 

.7247 

.3803 

.2412 

.25 

*Lx 




.7535 

.6967 

.6554 

.6003 

.4966 

.3983 

.3615 

.3060 


v ( e ^)max 




1.608 

1.542 

1.408 

1.180 

.9287 

.8147 

.4493 

.3128 

.10 

e 2 

max 





,4246 

.4067 

.3899 

.3635 

.3312 

.3153 

.2808 







.8632 

.8405 

.7459 

.6425 

.5259 

.3318 

.2668 

,05 

e 2 

max 


; 




.3276 

.3168 

.3056 

.2947 

.2881 

.2595 


v ( e2 )max 






.5294 

.5183 

.4802 

.3888 

.2646 

.2040 

.025 

¥ 2 







.2847 

.2726 

.2685 

.2616 

.2441 


max 













V <* 2 W 


| 





.2851 

.2593 

.2468 

.2127 

.1353 

.01 

®max 








.2378 

.2364 

.2352 

.2302 


V(e^)nax 


1 


1 




.1474 

.1423 

.1385 

.1073 

.005 

S* 

max 









.2353 

.2357 

.2302 


V(c^)max 









.1404 

.1369 

.1061 

.002 

5jLx 










.2229 

.2196 


^(® 2 ^max 










.0541 

.0460 

.001 

r2 | 

e max 











| .2195 


V( e ^)infl v 











.0460 


“trip - 0. In all other cases dp ^ 1, 



TABLE XV. - Continued. EFFECT OF u p AMD « f ON MEAN SQUARED ERROR AND ON VARIANCE OF MEAN SQUARED ERROR 
(£) 6 =1, i, 1 , 1; g = A; 6 = 1.0; t - 1.0; h = 0; n e » 1000 


a p 





1.00 

0.75 

0.50 

0.25 

0.10 

0.05 

0.025 

0.01 

0.005 

0.002 

0.001 

a 1.00 «max 

V (ft 2 ) max 

i>oo 

^ 2 W 

- 75 eLx 

V(^) m ax 

®raax 

v^U 

.25 e 2 
* c max 

V(e 2 ) max 
*10 4ax 

V(e 2 ) mx 

.05 ®max 

V (ft 2 ) max 

‘025 4, x 

v (®^max 

.01 ^roax 

v ( e2 )roax 

.005 e tnax 

^( ft2 ^niax 

.002 i^ x 

VCe^max 

*001 e 2 ^ 

V(F 2 ) 
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TABLE XV. - Concluded. EFFECT OF c* p AND ON MEAN SQUARED 

ERROR AND ON VARIANCE OF MEAN SQUARED ERROR 
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Figure 2. - Half normal plot. 



(a) SECTIONS 1, 2, AND 3 
Figure 3 - Flow chart for POOLES. 













(b) SECTIONS 4, 5, 6, 7, AND & 
Figured -Concluded. 
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ABSTRACT 


The investigation consisted of Monte Carlo studies using population 
models intended to represent response surface applications. Simulated 
experiments were generated by adding pseudo random normally distributed 
errors to population values to generate "observations." Model equations 
were fitted to the "observations" and the decision procedure was used to 
delete terms. Comparison of values predicted by the reduced models with 
the true population values enabled the identification of deletion strate- 
gies that are approximately optimal for minimizing prediction errors. 
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